[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:06 dist-test-slave-zkpd krb5kdc[2012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:06 dist-test-slave-zkpd krb5kdc[2012](info): set up 2 sockets
Aug 07 04:22:06 dist-test-slave-zkpd krb5kdc[2012](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 04:22:07.631249 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:07 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:07.807432 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45725
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:08.128209 2025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:08.128543 2025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:08.128695 2025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:08.135843 2025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:08.136008 2025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:08.136113 2025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:08.136210 2025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:08.136301 2025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:08.146112 2025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:08.147396 2025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:08.148717 2025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:08.160594 2031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:08.160671 2030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:08.160696 2033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:08.161500 2025 server_base.cc:1034] running on GCE node
I20260807 04:22:08.162230 2025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:08.163575 2025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:08.164815 2025 hybrid_clock.cc:648] HybridClock initialized: now 1786076528164743 us; error 78 us; skew 500 ppm
I20260807 04:22:08.165162 2025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:08 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076528, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:08.168313 2025 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:08.169704 2025 webserver.cc:460] Webserver started at http://127.1.244.254:34889/ using document root <none> and password file <none>
I20260807 04:22:08.170387 2025 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:08.170550 2025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:08.170864 2025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:08.174808 2025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "3180098b8003414884dd6907339e871f"
format_stamp: "Formatted at 2026-08-07 04:22:08 on dist-test-slave-zkpd"
server_key: "a71afd8ad5bf0f611d2ac676c4aaf03d"
server_key_iv: "5545e54f23cfb4c977f61e215b019561"
server_key_version: "encryptionkey@0"
I20260807 04:22:08.175582 2025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "3180098b8003414884dd6907339e871f"
format_stamp: "Formatted at 2026-08-07 04:22:08 on dist-test-slave-zkpd"
server_key: "a71afd8ad5bf0f611d2ac676c4aaf03d"
server_key_iv: "5545e54f23cfb4c977f61e215b019561"
server_key_version: "encryptionkey@0"
I20260807 04:22:08.180778 2025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:22:08.184196 2040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:08.185559 2025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:22:08.185761 2025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "3180098b8003414884dd6907339e871f"
format_stamp: "Formatted at 2026-08-07 04:22:08 on dist-test-slave-zkpd"
server_key: "a71afd8ad5bf0f611d2ac676c4aaf03d"
server_key_iv: "5545e54f23cfb4c977f61e215b019561"
server_key_version: "encryptionkey@0"
I20260807 04:22:08.185971 2025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:08.210937 2025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:08.215587 2025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:08.215952 2025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:08.234951 2025 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45725
I20260807 04:22:08.234969 2093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45725 every 8 connection(s)
I20260807 04:22:08.236372 2025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:08.240010 2094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:08.244278 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2025
I20260807 04:22:08.244644 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:08.246434 2003 external_mini_cluster.cc:1428] Setting key 8d30d7a0ff95254b3700ec5cee80da17
I20260807 04:22:08.250223 2094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f: Bootstrap starting.
I20260807 04:22:08.253741 2094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:08.254719 2094 log.cc:824] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:08.257314 2094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f: No bootstrap required, opened a new log
I20260807 04:22:08.265552 2094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3180098b8003414884dd6907339e871f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45725 } }
I20260807 04:22:08.265928 2094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:08.266053 2094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3180098b8003414884dd6907339e871f, State: Initialized, Role: FOLLOWER
I20260807 04:22:08.266585 2094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3180098b8003414884dd6907339e871f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45725 } }
I20260807 04:22:08.266834 2094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:08.266983 2094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:08.267165 2094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:08.269661 2094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3180098b8003414884dd6907339e871f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45725 } }
I20260807 04:22:08.270248 2094 leader_election.cc:302] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3180098b8003414884dd6907339e871f; no voters:
I20260807 04:22:08.270857 2094 leader_election.cc:288] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:08.271005 2099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:08.272120 2099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [term 1 LEADER]: Becoming Leader. State: Replica: 3180098b8003414884dd6907339e871f, State: Running, Role: LEADER
I20260807 04:22:08.272748 2099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3180098b8003414884dd6907339e871f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45725 } }
I20260807 04:22:08.273113 2094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 04:22:08 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:08.277395 2101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3180098b8003414884dd6907339e871f. Latest consensus state: current_term: 1 leader_uuid: "3180098b8003414884dd6907339e871f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3180098b8003414884dd6907339e871f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45725 } } }
I20260807 04:22:08.277845 2101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:08.278501 2100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3180098b8003414884dd6907339e871f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3180098b8003414884dd6907339e871f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45725 } } }
I20260807 04:22:08.278896 2100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:08.286774 2108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:08.292551 2096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:08.250182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.1.244.254:45725)
0807 04:22:08.251071 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:08.251088 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:08.252459 (+ 1371us) server_negotiation.cc:407] Connection header received
0807 04:22:08.261813 (+ 9354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:08.261832 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:08.262160 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:08.262544 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:08.265356 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:08.267013 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:08.268808 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:08.269848 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:08.270042 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:08.279154 (+ 9112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:08.279173 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:08.279200 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:08.279261 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:08.284494 (+ 5233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:08.285785 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:08.285798 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:08.285813 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:08.286093 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:08.286738 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:08.286744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:08.286750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:08.287588 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:08.287929 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:08.290554 (+ 2625us) server_negotiation.cc:299] Negotiation successful
0807 04:22:08.290961 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":384,"threads_started":1}
I20260807 04:22:08.293243 2108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:08.303385 2108 catalog_manager.cc:1347] Generated new cluster ID: e0d7bbd7a9054efba31dcdba7a1b14e3
I20260807 04:22:08.303516 2108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:08.320340 2108 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:08.321564 2108 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:08.327744 2108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3180098b8003414884dd6907339e871f: Generated new TSK 0
I20260807 04:22:08.328553 2108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:08.374939 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45725
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:08.704500 2121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:08.704757 2121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:08.704851 2121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:08.711738 2121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:08.711831 2121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:08.711959 2121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:08.721331 2121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:08.722491 2121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:08.723750 2121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:08.737025 2127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:08.737022 2129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:08.737023 2126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:08.737870 2121 server_base.cc:1034] running on GCE node
I20260807 04:22:08.738432 2121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:08.739305 2121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:08.740499 2121 hybrid_clock.cc:648] HybridClock initialized: now 1786076528740466 us; error 26 us; skew 500 ppm
I20260807 04:22:08.740800 2121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:08 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076528, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:08.744028 2121 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:08.745481 2121 webserver.cc:460] Webserver started at http://127.1.244.193:40735/ using document root <none> and password file <none>
I20260807 04:22:08.746160 2121 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:08.746289 2121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:08.746604 2121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:08.749425 2121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "88a9c7f04e1f4f1287b6b129aef60ee4"
format_stamp: "Formatted at 2026-08-07 04:22:08 on dist-test-slave-zkpd"
server_key: "720f7ae33b8530cac4d288fb67888a69"
server_key_iv: "221392673eb2aac6fc0735495280ee8d"
server_key_version: "encryptionkey@0"
I20260807 04:22:08.750193 2121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "88a9c7f04e1f4f1287b6b129aef60ee4"
format_stamp: "Formatted at 2026-08-07 04:22:08 on dist-test-slave-zkpd"
server_key: "720f7ae33b8530cac4d288fb67888a69"
server_key_iv: "221392673eb2aac6fc0735495280ee8d"
server_key_version: "encryptionkey@0"
I20260807 04:22:08.755056 2121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:22:08.758365 2136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:08.759582 2121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:22:08.759747 2121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "88a9c7f04e1f4f1287b6b129aef60ee4"
format_stamp: "Formatted at 2026-08-07 04:22:08 on dist-test-slave-zkpd"
server_key: "720f7ae33b8530cac4d288fb67888a69"
server_key_iv: "221392673eb2aac6fc0735495280ee8d"
server_key_version: "encryptionkey@0"
I20260807 04:22:08.759960 2121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:08.785507 2121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:08.790045 2121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:08.790335 2121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:08.791368 2121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:08.792737 2121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:08.792845 2121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:08.793013 2121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:08.793089 2121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:08.827662 2121 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34751
I20260807 04:22:08.827729 2251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34751 every 8 connection(s)
I20260807 04:22:08.829038 2121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:08.831319 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2121
I20260807 04:22:08.831559 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:08.832362 2003 external_mini_cluster.cc:1428] Setting key 582550c911af1ae0eef8a2d14da2a043
Aug 07 04:22:08 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076528, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:08.851701 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:08.831596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35825 (local address 127.1.244.254:45725)
0807 04:22:08.832129 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:08.832138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:08.833231 (+ 1093us) server_negotiation.cc:407] Connection header received
0807 04:22:08.835609 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:08.835613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:08.835672 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:08.835761 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:08.837898 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:08.838593 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:08.840835 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:08.841549 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:08.841700 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:08.844958 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:08.844966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:08.844973 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:08.845016 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:08.848611 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:08.849301 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:08.849309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:08.849316 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:08.849471 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:08.849880 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:08.849886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:08.849891 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:08.850255 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:08.850410 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:08.851034 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 04:22:08.851261 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":268,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260807 04:22:08.852274 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:08.832247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45725 (local address 127.1.244.193:35825)
0807 04:22:08.833063 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:08.833119 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:08.835313 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:08.836061 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:08.836074 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:08.836886 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:08.837724 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:08.837753 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:08.838740 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:08.838748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:08.840672 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:08.840688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:08.841759 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:08.841763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:08.841943 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:08.842614 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:08.842634 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:08.844743 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:08.848743 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:08.848749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:08.848760 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:08.849167 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:08.849589 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:08.849592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:08.849594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:08.849757 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:08.850376 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:08.850381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:08.850645 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 04:22:08.850844 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 04:22:08.851193 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":333,"threads_started":1}
I20260807 04:22:08.853840 2252 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45725
I20260807 04:22:08.854126 2252 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
I20260807 04:22:08.854991 2252 heartbeater.cc:511] Master 127.1.244.254:45725 requested a full tablet report, sending...
I20260807 04:22:08.857080 2057 ts_manager.cc:194] Registered new tserver with Master: 88a9c7f04e1f4f1287b6b129aef60ee4 (127.1.244.193:34751)
I20260807 04:22:08.858469 2057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:35825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:08.876828 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45725
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:09.200407 2259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:09.200711 2259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:09.200884 2259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:09.208091 2259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:09.208221 2259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:09.208388 2259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:09.217717 2259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:09.218932 2259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:09.220196 2259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:09.233006 2264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:09.232970 2267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:09.233074 2265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:09.234331 2259 server_base.cc:1034] running on GCE node
I20260807 04:22:09.235059 2259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:09.236168 2259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:09.237417 2259 hybrid_clock.cc:648] HybridClock initialized: now 1786076529237341 us; error 55 us; skew 500 ppm
I20260807 04:22:09.237763 2259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:09.241487 2259 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:09.249300 2259 webserver.cc:460] Webserver started at http://127.1.244.194:42303/ using document root <none> and password file <none>
I20260807 04:22:09.249967 2259 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:09.250120 2259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:09.250429 2259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:09.253315 2259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "fedf58f4e0814318866d0655d7776c2a"
format_stamp: "Formatted at 2026-08-07 04:22:09 on dist-test-slave-zkpd"
server_key: "aba563458cd555d43ff2e1eeea77ab7d"
server_key_iv: "082e72654bf375ff2c6954ad0f6a731c"
server_key_version: "encryptionkey@0"
I20260807 04:22:09.254074 2259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "fedf58f4e0814318866d0655d7776c2a"
format_stamp: "Formatted at 2026-08-07 04:22:09 on dist-test-slave-zkpd"
server_key: "aba563458cd555d43ff2e1eeea77ab7d"
server_key_iv: "082e72654bf375ff2c6954ad0f6a731c"
server_key_version: "encryptionkey@0"
I20260807 04:22:09.259029 2259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 04:22:09.262657 2275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:09.264220 2259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:09.264400 2259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "fedf58f4e0814318866d0655d7776c2a"
format_stamp: "Formatted at 2026-08-07 04:22:09 on dist-test-slave-zkpd"
server_key: "aba563458cd555d43ff2e1eeea77ab7d"
server_key_iv: "082e72654bf375ff2c6954ad0f6a731c"
server_key_version: "encryptionkey@0"
I20260807 04:22:09.264670 2259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:09.298765 2259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:09.303267 2259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:09.303561 2259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:09.304622 2259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:09.305970 2259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:09.306078 2259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:09.306238 2259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:09.306314 2259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:09.340515 2259 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:36907
I20260807 04:22:09.340598 2387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:36907 every 8 connection(s)
I20260807 04:22:09.341843 2259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:09.342834 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2259
I20260807 04:22:09.343065 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:09.343621 2003 external_mini_cluster.cc:1428] Setting key 818f496fa6ff7ffe15d8cbc4c05d8157
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260807 04:22:09.366551 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.344778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:46017 (local address 127.1.244.254:45725)
0807 04:22:09.344985 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:09.344991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:09.346453 (+ 1462us) server_negotiation.cc:407] Connection header received
0807 04:22:09.349512 (+ 3059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:09.349516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:09.349585 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:09.349666 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:09.351786 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.353015 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.354549 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.355341 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.355485 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.359561 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:09.359569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:09.359574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:09.359620 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:09.363356 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.364232 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.364238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.364242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.364393 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.364852 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.364859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.364864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.365163 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:09.365285 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:09.366088 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 04:22:09.366312 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
I20260807 04:22:09.367597 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.345364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45725 (local address 127.1.244.194:46017)
0807 04:22:09.346271 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:09.346320 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:09.349190 (+ 2870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:09.349947 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:09.349960 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:09.350776 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:09.351591 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:09.351619 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.353167 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.353170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:09.354403 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:09.354417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.355551 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.355555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:09.355740 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.356508 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:09.356548 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:09.359320 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:09.363492 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:09.363498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:09.363509 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:09.364002 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:09.364502 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:09.364505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:09.364507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:09.364732 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:09.365278 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:09.365283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:09.365694 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 04:22:09.365946 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 04:22:09.366318 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":392,"threads_started":1}
I20260807 04:22:09.369139 2388 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45725
I20260807 04:22:09.369462 2388 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:09.370167 2388 heartbeater.cc:511] Master 127.1.244.254:45725 requested a full tablet report, sending...
I20260807 04:22:09.371765 2057 ts_manager.cc:194] Registered new tserver with Master: fedf58f4e0814318866d0655d7776c2a (127.1.244.194:36907)
I20260807 04:22:09.373066 2057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:46017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:09.391283 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45725
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:09.722873 2394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:09.723136 2394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:09.723232 2394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:09.730226 2394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:09.730321 2394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:09.730427 2394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:09.740419 2394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45725
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:09.741496 2394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:09.742700 2394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:09.755326 2400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:09.755436 2399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:09.755702 2402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:09.756384 2394 server_base.cc:1034] running on GCE node
I20260807 04:22:09.756937 2394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:09.757925 2394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:09.759142 2394 hybrid_clock.cc:648] HybridClock initialized: now 1786076529759107 us; error 43 us; skew 500 ppm
I20260807 04:22:09.759438 2394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:09.762741 2394 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:09.764262 2394 webserver.cc:460] Webserver started at http://127.1.244.195:46265/ using document root <none> and password file <none>
I20260807 04:22:09.764923 2394 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:09.765020 2394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:09.765272 2394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:09.768115 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "5d527e97c6a643e78b4ac37c29b86eb3"
format_stamp: "Formatted at 2026-08-07 04:22:09 on dist-test-slave-zkpd"
server_key: "f395db5e479171674fb68d7a7f338e06"
server_key_iv: "036aed77b59477aac26524b2a7587b5f"
server_key_version: "encryptionkey@0"
I20260807 04:22:09.768834 2394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "5d527e97c6a643e78b4ac37c29b86eb3"
format_stamp: "Formatted at 2026-08-07 04:22:09 on dist-test-slave-zkpd"
server_key: "f395db5e479171674fb68d7a7f338e06"
server_key_iv: "036aed77b59477aac26524b2a7587b5f"
server_key_version: "encryptionkey@0"
I20260807 04:22:09.773805 2394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 04:22:09.777237 2410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:09.778470 2394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:22:09.778630 2394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "5d527e97c6a643e78b4ac37c29b86eb3"
format_stamp: "Formatted at 2026-08-07 04:22:09 on dist-test-slave-zkpd"
server_key: "f395db5e479171674fb68d7a7f338e06"
server_key_iv: "036aed77b59477aac26524b2a7587b5f"
server_key_version: "encryptionkey@0"
I20260807 04:22:09.778807 2394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:09.795499 2394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:09.799795 2394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:09.800110 2394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:09.801036 2394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:09.802309 2394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:09.802382 2394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:09.802531 2394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:09.802588 2394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:09.835858 2394 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:35365
I20260807 04:22:09.835991 2523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:35365 every 8 connection(s)
I20260807 04:22:09.837137 2394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:09.839057 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2394
I20260807 04:22:09.839253 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:09.839735 2003 external_mini_cluster.cc:1428] Setting key d9bff1746dbb5b4d659ca7505519a42c
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:09.856977 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.839919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:41261 (local address 127.1.244.254:45725)
0807 04:22:09.840128 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:09.840134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:09.841377 (+ 1243us) server_negotiation.cc:407] Connection header received
0807 04:22:09.843514 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:09.843520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:09.843643 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:09.843744 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:09.845251 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.845997 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.847444 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.847823 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.847971 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.851083 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:09.851088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:09.851091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:09.851120 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:09.853810 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.854606 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.854611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.854614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.854811 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.855303 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.855307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.855309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.855682 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:09.855851 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:09.856573 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 04:22:09.856729 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:22:09.857801 2526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.840401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45725 (local address 127.1.244.195:41261)
0807 04:22:09.841209 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:09.841266 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:09.843166 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:09.844004 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:09.844012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:09.844578 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:09.845113 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:09.845129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.846123 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.846126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:09.847321 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:09.847333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.848055 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.848060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:09.848246 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.848836 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:09.848861 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:09.850908 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:09.853945 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:09.853951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:09.853961 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:09.854482 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:09.854937 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:09.854942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:09.854945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:09.855178 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:09.855821 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:09.855830 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:09.856234 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 04:22:09.856455 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 04:22:09.856791 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":351,"threads_started":1}
I20260807 04:22:09.859261 2524 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45725
I20260807 04:22:09.859519 2524 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:09.860237 2524 heartbeater.cc:511] Master 127.1.244.254:45725 requested a full tablet report, sending...
I20260807 04:22:09.861264 2252 heartbeater.cc:503] Master 127.1.244.254:45725 was elected leader, sending a full tablet report...
I20260807 04:22:09.861635 2057 ts_manager.cc:194] Registered new tserver with Master: 5d527e97c6a643e78b4ac37c29b86eb3 (127.1.244.195:35365)
I20260807 04:22:09.862589 2057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:41261
I20260807 04:22:09.868120 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:09.914265 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.899062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.1.244.193:34751)
0807 04:22:09.900364 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:09.900372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:09.900397 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:22:09.900479 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:09.900485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:09.900706 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:09.900852 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:09.902212 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.903191 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.904496 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.905063 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.905159 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.908873 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:09.908885 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:09.908897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:09.908941 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:09.911795 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.912483 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.912489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.912491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.912610 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.912922 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.912925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.912927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.913239 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:09.913458 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:09.913664 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 04:22:09.913884 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1155,"thread_start_us":1010,"threads_started":1}
W20260807 04:22:09.915458 2166 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59362
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:22:09 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:09.958765 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.945258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.1.244.193:34751)
0807 04:22:09.945484 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:09.945491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:09.945706 (+ 215us) server_negotiation.cc:407] Connection header received
0807 04:22:09.945914 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:09.945919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:09.946018 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:09.946123 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:09.947485 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.948233 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.949503 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.949913 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.950049 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.953636 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:09.953641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:09.953644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:09.953675 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:09.956088 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.956719 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.956730 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.956732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.956848 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.957209 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.957214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.957217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.957474 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:09.957713 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:09.958359 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 04:22:09.958550 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:22:09.960276 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2121
I20260807 04:22:09.972322 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:09.961185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59378 (local address 127.1.244.193:34751)
0807 04:22:09.961335 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:09.961339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:09.961563 (+ 224us) server_negotiation.cc:407] Connection header received
0807 04:22:09.961663 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:09.961666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:09.961712 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:09.961781 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:09.962845 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.963596 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.964899 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:09.965341 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:09.965439 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:09.967467 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:09.967471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:09.967474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:09.967500 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:09.970107 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.970698 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.970701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.970703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.970842 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:09.971262 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:09.971265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:09.971267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:09.971504 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:09.971661 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:09.971946 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 04:22:09.972088 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 04:22:09.972968 2247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:10.056653 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2121
I20260807 04:22:10.075922 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2259
Aug 07 04:22:10 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:10.091873 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:10.076920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.1.244.194:36907)
0807 04:22:10.077472 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:10.077479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:10.077503 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:22:10.077570 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:10.077575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:10.077779 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:10.077909 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:10.079073 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:10.080124 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:10.081329 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:10.081873 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:10.082006 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:10.086069 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:10.086079 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:10.086093 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:10.086123 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:10.089372 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:10.090047 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:10.090054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:10.090056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:10.090172 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:10.090567 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:10.090570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:10.090572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:10.090897 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:10.091111 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:10.091374 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 04:22:10.091571 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
I20260807 04:22:10.093295 2383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:10.176286 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2259
I20260807 04:22:10.194921 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2394
Aug 07 04:22:10 dist-test-slave-zkpd krb5kdc[2012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:10.210899 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:10.195786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.1.244.195:35365)
0807 04:22:10.196498 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:10.196505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:10.196535 (+ 30us) server_negotiation.cc:407] Connection header received
0807 04:22:10.196598 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:10.196605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:10.196815 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:10.196949 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:10.198135 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:10.199147 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:10.200566 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:10.201101 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:10.201195 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:10.205186 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:10.205198 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:10.205211 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:10.205244 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:10.208160 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:10.208842 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:10.208850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:10.208852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:10.208971 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:10.209348 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:10.209352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:10.209354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:10.209679 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:10.210009 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:10.210307 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 04:22:10.210574 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":366,"threads_started":1}
I20260807 04:22:10.212194 2519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:10.295328 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2394
I20260807 04:22:10.313359 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2025
I20260807 04:22:10.314584 2088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:10.416795 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2025
2026-08-07T04:22:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3922 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:10 dist-test-slave-zkpd krb5kdc[2547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:10 dist-test-slave-zkpd krb5kdc[2547](info): set up 2 sockets
Aug 07 04:22:10 dist-test-slave-zkpd krb5kdc[2547](info): commencing operation
krb5kdc: starting...
W20260807 04:22:11.506282 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:11 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:11.689175 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:32943
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:32943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:12.010818 2560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:12.011078 2560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:12.011164 2560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:12.018455 2560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:12.018548 2560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:12.018594 2560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:12.018672 2560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:12.018757 2560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:12.028363 2560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:32943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:32943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:12.029639 2560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:12.030948 2560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:12.043038 2566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:12.043026 2568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:12.043061 2565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:12.043947 2560 server_base.cc:1034] running on GCE node
I20260807 04:22:12.044719 2560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:12.046031 2560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:12.047287 2560 hybrid_clock.cc:648] HybridClock initialized: now 1786076532047236 us; error 36 us; skew 500 ppm
I20260807 04:22:12.047708 2560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:12 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076532, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:12.055617 2560 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:12.057315 2560 webserver.cc:460] Webserver started at http://127.1.244.254:34837/ using document root <none> and password file <none>
I20260807 04:22:12.058017 2560 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:12.058127 2560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:12.058418 2560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:12.061748 2560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "73f9dec8aea641048c29f9224034a938"
format_stamp: "Formatted at 2026-08-07 04:22:12 on dist-test-slave-zkpd"
server_key: "5e775ca3bbaf159dd78345e0d972e432"
server_key_iv: "0436dcda4e38183bf369ebaf98fc1e41"
server_key_version: "encryptionkey@0"
I20260807 04:22:12.062525 2560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "73f9dec8aea641048c29f9224034a938"
format_stamp: "Formatted at 2026-08-07 04:22:12 on dist-test-slave-zkpd"
server_key: "5e775ca3bbaf159dd78345e0d972e432"
server_key_iv: "0436dcda4e38183bf369ebaf98fc1e41"
server_key_version: "encryptionkey@0"
I20260807 04:22:12.068167 2560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:22:12.071619 2576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:12.072970 2560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:12.073149 2560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "73f9dec8aea641048c29f9224034a938"
format_stamp: "Formatted at 2026-08-07 04:22:12 on dist-test-slave-zkpd"
server_key: "5e775ca3bbaf159dd78345e0d972e432"
server_key_iv: "0436dcda4e38183bf369ebaf98fc1e41"
server_key_version: "encryptionkey@0"
I20260807 04:22:12.073341 2560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:12.090989 2560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:12.095211 2560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:12.095492 2560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:12.113570 2560 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:32943
I20260807 04:22:12.113592 2627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:32943 every 8 connection(s)
I20260807 04:22:12.114981 2560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:12.115530 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2560
I20260807 04:22:12.115772 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:12.116602 2003 external_mini_cluster.cc:1428] Setting key 745d768991853fb7fda96fcaf358ce18
I20260807 04:22:12.119361 2628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:12.129962 2628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938: Bootstrap starting.
Aug 07 04:22:12 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:12.133729 2628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:12.135203 2628 log.cc:824] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:12.138785 2628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938: No bootstrap required, opened a new log
I20260807 04:22:12.140148 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:12.118606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.1.244.254:32943)
0807 04:22:12.119372 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:12.119384 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:12.119452 (+ 68us) server_negotiation.cc:407] Connection header received
0807 04:22:12.121210 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:12.121231 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:12.121594 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:12.121995 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:12.123459 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:12.124878 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:12.126102 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:12.126762 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:12.126990 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:12.131165 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:12.131173 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:12.131186 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:12.131226 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:12.135570 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:12.136413 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:12.136419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:12.136425 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:12.136589 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:12.137014 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:12.137019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:12.137022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:12.137666 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:12.137949 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:12.138360 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 04:22:12.138773 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":362,"threads_started":1}
I20260807 04:22:12.147290 2628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f9dec8aea641048c29f9224034a938" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32943 } }
I20260807 04:22:12.147608 2628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:12.147682 2628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f9dec8aea641048c29f9224034a938, State: Initialized, Role: FOLLOWER
I20260807 04:22:12.148185 2628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f9dec8aea641048c29f9224034a938" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32943 } }
I20260807 04:22:12.148379 2628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:12.148474 2628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:12.148591 2628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:12.150538 2628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f9dec8aea641048c29f9224034a938" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32943 } }
I20260807 04:22:12.150940 2628 leader_election.cc:302] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73f9dec8aea641048c29f9224034a938; no voters:
I20260807 04:22:12.151539 2628 leader_election.cc:288] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:12.151649 2633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:12.152391 2633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [term 1 LEADER]: Becoming Leader. State: Replica: 73f9dec8aea641048c29f9224034a938, State: Running, Role: LEADER
I20260807 04:22:12.152958 2633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f9dec8aea641048c29f9224034a938" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32943 } }
I20260807 04:22:12.153630 2628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:12.155928 2634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73f9dec8aea641048c29f9224034a938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f9dec8aea641048c29f9224034a938" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32943 } } }
I20260807 04:22:12.155879 2635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73f9dec8aea641048c29f9224034a938. Latest consensus state: current_term: 1 leader_uuid: "73f9dec8aea641048c29f9224034a938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f9dec8aea641048c29f9224034a938" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32943 } } }
I20260807 04:22:12.156334 2634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:12.156334 2635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:12.157928 2640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:12.163316 2640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:12.172482 2640 catalog_manager.cc:1347] Generated new cluster ID: ae319c653d0b4c8b8e6d3db586faa895
I20260807 04:22:12.172595 2640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:12.188166 2640 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:12.189383 2640 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:12.194617 2640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73f9dec8aea641048c29f9224034a938: Generated new TSK 0
I20260807 04:22:12.195318 2640 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:12.253341 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:32943
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:12.595112 2654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:12.595427 2654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:12.595602 2654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:12.602773 2654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:12.602905 2654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:12.603089 2654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:12.612669 2654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:32943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:12.613960 2654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:12.615305 2654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:12.628320 2662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:12.628290 2660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:12.628696 2659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:12.629176 2654 server_base.cc:1034] running on GCE node
I20260807 04:22:12.629803 2654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:12.630827 2654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:12.632052 2654 hybrid_clock.cc:648] HybridClock initialized: now 1786076532632001 us; error 38 us; skew 500 ppm
I20260807 04:22:12.632369 2654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:12 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076532, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:12.635387 2654 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:12.636916 2654 webserver.cc:460] Webserver started at http://127.1.244.193:37499/ using document root <none> and password file <none>
I20260807 04:22:12.637620 2654 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:12.637719 2654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:12.637964 2654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:12.641000 2654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "4c16f36da5d14092af23134dd5a5b523"
format_stamp: "Formatted at 2026-08-07 04:22:12 on dist-test-slave-zkpd"
server_key: "b6d12b3095f0a598ffdc669e9c8ee595"
server_key_iv: "4fd01cf52464cea9f6cec3b8f6f82d19"
server_key_version: "encryptionkey@0"
I20260807 04:22:12.641676 2654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "4c16f36da5d14092af23134dd5a5b523"
format_stamp: "Formatted at 2026-08-07 04:22:12 on dist-test-slave-zkpd"
server_key: "b6d12b3095f0a598ffdc669e9c8ee595"
server_key_iv: "4fd01cf52464cea9f6cec3b8f6f82d19"
server_key_version: "encryptionkey@0"
I20260807 04:22:12.646503 2654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:12.649775 2669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:12.650977 2654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:22:12.651096 2654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "4c16f36da5d14092af23134dd5a5b523"
format_stamp: "Formatted at 2026-08-07 04:22:12 on dist-test-slave-zkpd"
server_key: "b6d12b3095f0a598ffdc669e9c8ee595"
server_key_iv: "4fd01cf52464cea9f6cec3b8f6f82d19"
server_key_version: "encryptionkey@0"
I20260807 04:22:12.651254 2654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:12.666360 2654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:12.669970 2654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:12.670186 2654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:12.670995 2654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:12.672215 2654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:12.672287 2654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:12.672372 2654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:12.672425 2654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:12.705147 2654 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:40007
I20260807 04:22:12.705202 2784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:40007 every 8 connection(s)
I20260807 04:22:12.706455 2654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:12.713754 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2654
I20260807 04:22:12.713969 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:12.714587 2003 external_mini_cluster.cc:1428] Setting key 9cfb011abfda8fb2d5f64cb4b6a4cfbf
Aug 07 04:22:12 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076532, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:12.726293 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:12.708918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53635 (local address 127.1.244.254:32943)
0807 04:22:12.709327 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:12.709331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:12.710156 (+ 825us) server_negotiation.cc:407] Connection header received
0807 04:22:12.711918 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:12.711923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:12.712023 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:12.712142 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:12.713572 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:12.714378 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:12.716160 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:12.716668 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:12.716763 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:12.720721 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:12.720726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:12.720730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:12.720760 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:12.723260 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:12.723993 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:12.723999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:12.724001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:12.724123 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:12.724575 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:12.724579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:12.724581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:12.724820 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:12.724974 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:12.725679 (+ 705us) server_negotiation.cc:299] Negotiation successful
0807 04:22:12.725927 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":228,"threads_started":1}
I20260807 04:22:12.727224 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:12.709325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:32943 (local address 127.1.244.193:53635)
0807 04:22:12.710001 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:12.710043 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:12.711648 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:12.712367 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:12.712374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:12.712859 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:12.713412 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:12.713429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:12.714551 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:12.714555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:12.716010 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:12.716024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:12.716916 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:12.716925 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:12.717217 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:12.717916 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:12.717951 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:12.720510 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:12.723404 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:12.723413 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:12.723427 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:12.723824 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:12.724239 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:12.724244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:12.724248 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:12.724449 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:12.724956 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:12.724963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:12.725282 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 04:22:12.725574 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 04:22:12.726106 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":253,"threads_started":1}
I20260807 04:22:12.728734 2785 heartbeater.cc:347] Connected to a master server at 127.1.244.254:32943
I20260807 04:22:12.729027 2785 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:12.729722 2785 heartbeater.cc:511] Master 127.1.244.254:32943 requested a full tablet report, sending...
I20260807 04:22:12.731981 2593 ts_manager.cc:194] Registered new tserver with Master: 4c16f36da5d14092af23134dd5a5b523 (127.1.244.193:40007)
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260807 04:22:12.733901 2593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:53635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:12.756614 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:32943
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:13.075434 2791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:13.075752 2791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:13.075947 2791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:13.083024 2791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:13.083168 2791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:13.083376 2791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:13.092801 2791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:32943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:13.094089 2791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:13.095424 2791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:13.108898 2799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:13.108938 2797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:13.109002 2796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:13.109800 2791 server_base.cc:1034] running on GCE node
I20260807 04:22:13.110406 2791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:13.111289 2791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:13.112483 2791 hybrid_clock.cc:648] HybridClock initialized: now 1786076533112434 us; error 37 us; skew 500 ppm
I20260807 04:22:13.112779 2791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:13 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076533, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:13.116225 2791 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:13.117770 2791 webserver.cc:460] Webserver started at http://127.1.244.194:45735/ using document root <none> and password file <none>
I20260807 04:22:13.118619 2791 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:13.118746 2791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:13.119056 2791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:13.122792 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "7eef6b063f28458ea843dab5e1f08b0c"
format_stamp: "Formatted at 2026-08-07 04:22:13 on dist-test-slave-zkpd"
server_key: "0cad0a5d2d5949ca4995dba95faefb98"
server_key_iv: "2f6b6f0dac527fe8ff2ee48aa623ffa9"
server_key_version: "encryptionkey@0"
I20260807 04:22:13.123724 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "7eef6b063f28458ea843dab5e1f08b0c"
format_stamp: "Formatted at 2026-08-07 04:22:13 on dist-test-slave-zkpd"
server_key: "0cad0a5d2d5949ca4995dba95faefb98"
server_key_iv: "2f6b6f0dac527fe8ff2ee48aa623ffa9"
server_key_version: "encryptionkey@0"
I20260807 04:22:13.129837 2791 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 04:22:13.134059 2806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:13.135355 2791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:22:13.135535 2791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "7eef6b063f28458ea843dab5e1f08b0c"
format_stamp: "Formatted at 2026-08-07 04:22:13 on dist-test-slave-zkpd"
server_key: "0cad0a5d2d5949ca4995dba95faefb98"
server_key_iv: "2f6b6f0dac527fe8ff2ee48aa623ffa9"
server_key_version: "encryptionkey@0"
I20260807 04:22:13.135793 2791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:13.155177 2791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:13.159153 2791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:13.159390 2791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:13.160312 2791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:13.161504 2791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:13.161576 2791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:13.161667 2791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:13.161711 2791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:13.194607 2791 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:36759
I20260807 04:22:13.194674 2921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:36759 every 8 connection(s)
I20260807 04:22:13.195999 2791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:13.203824 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2791
I20260807 04:22:13.204058 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:13.204674 2003 external_mini_cluster.cc:1428] Setting key 26872077077363e063bff1837584d1b2
Aug 07 04:22:13 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076533, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:13.216954 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.198434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49399 (local address 127.1.244.254:32943)
0807 04:22:13.198615 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.198619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.199675 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 04:22:13.201527 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.201532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.201626 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.201763 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:13.203249 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.204088 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.206224 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.206890 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.207010 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.211169 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:13.211174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:13.211177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:13.211209 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:13.213854 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.214578 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.214591 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.214594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.214730 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.215166 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.215172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.215175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.215459 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:13.215626 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.216373 (+ 747us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.216607 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:22:13.217891 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.198832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:32943 (local address 127.1.244.194:49399)
0807 04:22:13.199501 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.199557 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.201263 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.202013 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.202021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.202533 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:13.203078 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.203095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.204281 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.204288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.206088 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.206100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.207135 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.207143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.207453 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.208272 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:13.208313 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:13.210952 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:13.214001 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:13.214010 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:13.214025 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:13.214450 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:13.214844 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:13.214849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:13.214853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:13.215053 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:13.215600 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:13.215608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:13.216009 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.216284 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.216796 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":246,"threads_started":1}
I20260807 04:22:13.219453 2922 heartbeater.cc:347] Connected to a master server at 127.1.244.254:32943
I20260807 04:22:13.219766 2922 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:13.220491 2922 heartbeater.cc:511] Master 127.1.244.254:32943 requested a full tablet report, sending...
I20260807 04:22:13.221928 2593 ts_manager.cc:194] Registered new tserver with Master: 7eef6b063f28458ea843dab5e1f08b0c (127.1.244.194:36759)
I20260807 04:22:13.222697 2593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:49399
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:13.246937 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:32943
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:13.564325 2927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:13.564576 2927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:13.564669 2927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:13.571491 2927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:13.571583 2927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:13.571692 2927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:13.581023 2927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:32943
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:13.582149 2927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:13.583360 2927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:13.596275 2932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:13.596451 2935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:13.596817 2933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:13.596853 2927 server_base.cc:1034] running on GCE node
I20260807 04:22:13.597546 2927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:13.598465 2927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:13.599658 2927 hybrid_clock.cc:648] HybridClock initialized: now 1786076533599622 us; error 26 us; skew 500 ppm
I20260807 04:22:13.599977 2927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:13 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076533, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:13.603029 2927 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:13.604513 2927 webserver.cc:460] Webserver started at http://127.1.244.195:40219/ using document root <none> and password file <none>
I20260807 04:22:13.605190 2927 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:13.605334 2927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:13.605650 2927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:13.608623 2927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "3846700b5c8f420db9a410860f80c446"
format_stamp: "Formatted at 2026-08-07 04:22:13 on dist-test-slave-zkpd"
server_key: "03c625249b09c16f110e42f3f161da37"
server_key_iv: "8b0c82a1370bb5535545c811cb03ee34"
server_key_version: "encryptionkey@0"
I20260807 04:22:13.609402 2927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "3846700b5c8f420db9a410860f80c446"
format_stamp: "Formatted at 2026-08-07 04:22:13 on dist-test-slave-zkpd"
server_key: "03c625249b09c16f110e42f3f161da37"
server_key_iv: "8b0c82a1370bb5535545c811cb03ee34"
server_key_version: "encryptionkey@0"
I20260807 04:22:13.614385 2927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:13.617660 2942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:13.618854 2927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 04:22:13.619036 2927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "3846700b5c8f420db9a410860f80c446"
format_stamp: "Formatted at 2026-08-07 04:22:13 on dist-test-slave-zkpd"
server_key: "03c625249b09c16f110e42f3f161da37"
server_key_iv: "8b0c82a1370bb5535545c811cb03ee34"
server_key_version: "encryptionkey@0"
I20260807 04:22:13.619236 2927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:13.651070 2927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:13.655444 2927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:13.655730 2927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:13.656739 2927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:13.658119 2927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:13.658218 2927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:13.658368 2927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:13.658443 2927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:13.691768 2927 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:43337
I20260807 04:22:13.691854 3057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:43337 every 8 connection(s)
I20260807 04:22:13.693121 2927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:13.694561 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 2927
I20260807 04:22:13.694765 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:13.695317 2003 external_mini_cluster.cc:1428] Setting key 29ec0f0eb123eb453b2468d9db4bf01d
Aug 07 04:22:13 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076533, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:13.714340 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.695806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56865 (local address 127.1.244.254:32943)
0807 04:22:13.696023 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.696028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.697453 (+ 1425us) server_negotiation.cc:407] Connection header received
0807 04:22:13.699529 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.699532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.699625 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.699704 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:13.701132 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.701856 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.704018 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.704546 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.704693 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.708307 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:13.708314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:13.708319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:13.708357 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:13.711737 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.712445 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.712449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.712451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.712583 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.712947 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.712950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.712952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.713219 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:13.713329 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.713943 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.714098 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:22:13.715207 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.696405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:32943 (local address 127.1.244.195:56865)
0807 04:22:13.697248 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.697293 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.699292 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.699870 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.699877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.700446 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:13.700990 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.701008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.702007 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.702011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.703820 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.703833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.704800 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.704807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.705007 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.705701 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:13.705728 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:13.708097 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:13.711856 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:13.711863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:13.711873 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:13.712326 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:13.712676 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:13.712678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:13.712681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:13.712848 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:13.713323 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:13.713329 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:13.713615 (+ 286us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.713840 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.714179 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":356,"threads_started":1}
I20260807 04:22:13.716594 3058 heartbeater.cc:347] Connected to a master server at 127.1.244.254:32943
I20260807 04:22:13.716878 3058 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:13.717589 3058 heartbeater.cc:511] Master 127.1.244.254:32943 requested a full tablet report, sending...
I20260807 04:22:13.718999 2593 ts_manager.cc:194] Registered new tserver with Master: 3846700b5c8f420db9a410860f80c446 (127.1.244.195:43337)
I20260807 04:22:13.719779 2593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:56865
I20260807 04:22:13.723038 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:22:13.736773 2785 heartbeater.cc:503] Master 127.1.244.254:32943 was elected leader, sending a full tablet report...
I20260807 04:22:13.742249 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.729678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49244 (local address 127.1.244.254:32943)
0807 04:22:13.729905 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.729912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.730309 (+ 397us) server_negotiation.cc:407] Connection header received
0807 04:22:13.730491 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.730496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.730559 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.730655 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:13.732143 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.732839 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.734137 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.734550 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.734685 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.737066 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:13.737073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:13.737076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:13.737112 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:13.739695 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.740472 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.740481 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.740484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.740605 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.741012 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.741017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.741022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.741338 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:13.741587 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.741885 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.742032 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:22:13.748018 2593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:13.754766 2593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:13.779481 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.767298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.1.244.194:36759)
0807 04:22:13.770823 (+ 3525us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.770835 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.770871 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:22:13.771002 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.771013 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.771856 (+ 843us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.773006 (+ 1150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:13.773279 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.774799 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.777243 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.778354 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.778515 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.778732 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.779017 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.779175 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3313,"thread_start_us":3182,"threads_started":1}
I20260807 04:22:13.779640 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.766947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36759 (local address 127.0.0.1:34722)
0807 04:22:13.769378 (+ 2431us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.769402 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.769554 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.772153 (+ 2599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.772161 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.772197 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:13.772747 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.772761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.774964 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.774970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.777082 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.777099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.778572 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.778579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.778777 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.778824 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.778954 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.779150 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2269,"mutex_wait_us":51,"thread_start_us":2116,"threads_started":1}
I20260807 04:22:13.780436 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.769524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43337 (local address 127.0.0.1:49458)
0807 04:22:13.770115 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.770134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.770249 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.771865 (+ 1616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.771872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.771924 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:13.772301 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.772312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.773888 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.773896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.776841 (+ 2945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.776862 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.779450 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.779463 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.779694 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.779743 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.779822 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.780007 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":24,"thread_start_us":1072,"threads_started":1}
I20260807 04:22:13.781239 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.770278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.1.244.195:43337)
0807 04:22:13.771220 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.771229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.771262 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:22:13.771360 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.771369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.771622 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.772474 (+ 852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:13.772669 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.773748 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.777012 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.778677 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.778799 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.779296 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.780707 (+ 1411us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.780872 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":590,"threads_started":1}
I20260807 04:22:13.782420 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.766757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:40007 (local address 127.0.0.1:52260)
0807 04:22:13.767750 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.767808 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.768037 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.774973 (+ 6936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.774978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.775006 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:13.775573 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.775586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.777577 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.777582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.779649 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.779663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.781619 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.781626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.781794 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.781831 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.781936 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.782058 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":43,"thread_start_us":715,"threads_started":1}
I20260807 04:22:13.782460 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.766976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52260 (local address 127.1.244.193:40007)
0807 04:22:13.774402 (+ 7426us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.774411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.774437 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:22:13.774532 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.774538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.774755 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.775758 (+ 1003us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:13.776028 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.777422 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.779827 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.781371 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.781570 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.781791 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.781977 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.782132 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7258,"thread_start_us":7077,"threads_started":1}
I20260807 04:22:13.784282 2856 tablet_service.cc:1467] Processing CreateTablet for tablet 876ca89484924bb9ba4d0a232158eed9 (DEFAULT_TABLE table=test-table [id=0168eaa6b03d4d29a6d27741b1ab5712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:13.784941 2991 tablet_service.cc:1467] Processing CreateTablet for tablet 876ca89484924bb9ba4d0a232158eed9 (DEFAULT_TABLE table=test-table [id=0168eaa6b03d4d29a6d27741b1ab5712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:13.786512 2856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 876ca89484924bb9ba4d0a232158eed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:13.786509 2991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 876ca89484924bb9ba4d0a232158eed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:13.786885 2719 tablet_service.cc:1467] Processing CreateTablet for tablet 876ca89484924bb9ba4d0a232158eed9 (DEFAULT_TABLE table=test-table [id=0168eaa6b03d4d29a6d27741b1ab5712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:13.788012 2719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 876ca89484924bb9ba4d0a232158eed9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:13.799153 3076 tablet_bootstrap.cc:492] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: Bootstrap starting.
I20260807 04:22:13.801313 3077 tablet_bootstrap.cc:492] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: Bootstrap starting.
I20260807 04:22:13.802803 3076 tablet_bootstrap.cc:654] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:13.802855 3078 tablet_bootstrap.cc:492] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: Bootstrap starting.
I20260807 04:22:13.804234 3076 log.cc:824] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:13.805027 3077 tablet_bootstrap.cc:654] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:13.805995 3078 tablet_bootstrap.cc:654] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:13.806192 3077 log.cc:824] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:13.807170 3078 log.cc:824] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:13.807629 3076 tablet_bootstrap.cc:492] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: No bootstrap required, opened a new log
I20260807 04:22:13.808066 3076 ts_tablet_manager.cc:1397] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 04:22:13.810319 3078 tablet_bootstrap.cc:492] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: No bootstrap required, opened a new log
I20260807 04:22:13.810581 3078 ts_tablet_manager.cc:1397] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260807 04:22:13.811143 3077 tablet_bootstrap.cc:492] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: No bootstrap required, opened a new log
I20260807 04:22:13.811399 3077 ts_tablet_manager.cc:1397] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 04:22:13.817481 3078 raft_consensus.cc:348] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.817950 3078 raft_consensus.cc:374] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:13.818076 3078 raft_consensus.cc:729] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eef6b063f28458ea843dab5e1f08b0c, State: Initialized, Role: FOLLOWER
I20260807 04:22:13.818007 3076 raft_consensus.cc:348] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.818441 3076 raft_consensus.cc:374] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:13.818590 3076 raft_consensus.cc:729] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c16f36da5d14092af23134dd5a5b523, State: Initialized, Role: FOLLOWER
I20260807 04:22:13.818750 3078 consensus_queue.cc:260] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.819263 3076 consensus_queue.cc:260] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.819911 2922 heartbeater.cc:503] Master 127.1.244.254:32943 was elected leader, sending a full tablet report...
I20260807 04:22:13.820544 3078 ts_tablet_manager.cc:1428] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260807 04:22:13.822361 3077 raft_consensus.cc:348] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.822906 3077 raft_consensus.cc:374] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:13.823060 3077 raft_consensus.cc:729] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3846700b5c8f420db9a410860f80c446, State: Initialized, Role: FOLLOWER
I20260807 04:22:13.823390 3076 ts_tablet_manager.cc:1428] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
I20260807 04:22:13.823707 3077 consensus_queue.cc:260] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.825982 3077 ts_tablet_manager.cc:1428] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260807 04:22:13.826385 3058 heartbeater.cc:503] Master 127.1.244.254:32943 was elected leader, sending a full tablet report...
I20260807 04:22:13.895334 3082 raft_consensus.cc:482] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:13.895660 3082 raft_consensus.cc:504] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.897377 3082 leader_election.cc:288] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c16f36da5d14092af23134dd5a5b523 (127.1.244.193:40007), 3846700b5c8f420db9a410860f80c446 (127.1.244.195:43337)
I20260807 04:22:13.904104 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.897907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35337 (local address 127.1.244.193:40007)
0807 04:22:13.898162 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.898171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.898857 (+ 686us) server_negotiation.cc:407] Connection header received
0807 04:22:13.899086 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.899090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.899150 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.899247 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:13.899802 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.900589 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.902343 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.903132 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.903305 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.903385 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.903700 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.903820 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 04:22:13.904235 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.897914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55123 (local address 127.1.244.195:43337)
0807 04:22:13.898159 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.898168 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.898962 (+ 794us) server_negotiation.cc:407] Connection header received
0807 04:22:13.899172 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.899176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.899234 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.899334 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:13.900024 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.900756 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.902477 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.903240 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.903426 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.903506 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.903826 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.903986 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 04:22:13.904345 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.897926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:43337 (local address 127.1.244.194:55123)
0807 04:22:13.898844 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.898858 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.899013 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.899356 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.899360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.899399 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:13.899826 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.899834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.900911 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.900916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.902316 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.902328 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.903480 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.903486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.903667 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.903689 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.903797 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.903933 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":2,"thread_start_us":570,"threads_started":1}
I20260807 04:22:13.904742 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.897796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:40007 (local address 127.1.244.194:35337)
0807 04:22:13.898741 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:13.898761 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:13.898934 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:13.899263 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:13.899267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:13.899289 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:13.899652 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.899664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.900715 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.900723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.902191 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:13.902202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.903353 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.903359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:13.903523 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.903546 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:22:13.903659 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 04:22:13.903779 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":4,"thread_start_us":602,"threads_started":1}
I20260807 04:22:13.905584 3011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876ca89484924bb9ba4d0a232158eed9" candidate_uuid: "7eef6b063f28458ea843dab5e1f08b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3846700b5c8f420db9a410860f80c446" is_pre_election: true
I20260807 04:22:13.905696 2739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876ca89484924bb9ba4d0a232158eed9" candidate_uuid: "7eef6b063f28458ea843dab5e1f08b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c16f36da5d14092af23134dd5a5b523" is_pre_election: true
I20260807 04:22:13.906126 3011 raft_consensus.cc:2454] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eef6b063f28458ea843dab5e1f08b0c in term 0.
I20260807 04:22:13.906199 2739 raft_consensus.cc:2454] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7eef6b063f28458ea843dab5e1f08b0c in term 0.
I20260807 04:22:13.906854 2810 leader_election.cc:302] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c16f36da5d14092af23134dd5a5b523, 7eef6b063f28458ea843dab5e1f08b0c; no voters:
I20260807 04:22:13.907232 3082 raft_consensus.cc:2781] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:13.907377 3082 raft_consensus.cc:482] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:13.907513 3082 raft_consensus.cc:3037] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:13.909940 3082 raft_consensus.cc:504] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.910605 3082 leader_election.cc:288] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [CANDIDATE]: Term 1 election: Requested vote from peers 4c16f36da5d14092af23134dd5a5b523 (127.1.244.193:40007), 3846700b5c8f420db9a410860f80c446 (127.1.244.195:43337)
I20260807 04:22:13.910990 2739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876ca89484924bb9ba4d0a232158eed9" candidate_uuid: "7eef6b063f28458ea843dab5e1f08b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c16f36da5d14092af23134dd5a5b523"
I20260807 04:22:13.911064 3011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876ca89484924bb9ba4d0a232158eed9" candidate_uuid: "7eef6b063f28458ea843dab5e1f08b0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3846700b5c8f420db9a410860f80c446"
I20260807 04:22:13.911211 2739 raft_consensus.cc:3037] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:13.911250 3011 raft_consensus.cc:3037] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:13.914404 3011 raft_consensus.cc:2454] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eef6b063f28458ea843dab5e1f08b0c in term 1.
I20260807 04:22:13.914780 2808 leader_election.cc:302] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3846700b5c8f420db9a410860f80c446, 7eef6b063f28458ea843dab5e1f08b0c; no voters:
I20260807 04:22:13.915114 3082 raft_consensus.cc:2781] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:13.915132 2739 raft_consensus.cc:2454] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7eef6b063f28458ea843dab5e1f08b0c in term 1.
I20260807 04:22:13.915648 3082 raft_consensus.cc:686] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [term 1 LEADER]: Becoming Leader. State: Replica: 7eef6b063f28458ea843dab5e1f08b0c, State: Running, Role: LEADER
I20260807 04:22:13.916188 3082 consensus_queue.cc:237] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } }
I20260807 04:22:13.920885 2592 catalog_manager.cc:5495] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7eef6b063f28458ea843dab5e1f08b0c (127.1.244.194). New cstate: current_term: 1 leader_uuid: "7eef6b063f28458ea843dab5e1f08b0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c16f36da5d14092af23134dd5a5b523" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7eef6b063f28458ea843dab5e1f08b0c" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3846700b5c8f420db9a410860f80c446" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 43337 } health_report { overall_health: UNKNOWN } } }
W20260807 04:22:13.944797 3059 tablet.cc:2369] T 876ca89484924bb9ba4d0a232158eed9 P 3846700b5c8f420db9a410860f80c446: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:22:13.947829 2923 tablet.cc:2369] T 876ca89484924bb9ba4d0a232158eed9 P 7eef6b063f28458ea843dab5e1f08b0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:22:13.958748 2786 tablet.cc:2369] T 876ca89484924bb9ba4d0a232158eed9 P 4c16f36da5d14092af23134dd5a5b523: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 04:22:13 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:13.974354 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:13.959946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.1.244.193:40007)
0807 04:22:13.960105 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:13.960109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:13.960639 (+ 530us) server_negotiation.cc:407] Connection header received
0807 04:22:13.960841 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:13.960846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:13.960928 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:13.961044 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:13.962540 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.963305 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.964636 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:13.965088 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:13.965232 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:13.969203 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:13.969216 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:13.969232 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:13.969261 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:13.972183 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.972769 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.972776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.972778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.972896 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:13.973198 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:13.973201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:13.973203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:13.973535 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:13.973763 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:13.973957 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 04:22:13.974105 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 04:22:13 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076533, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076533, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:14.021938 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:14.008059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.1.244.193:40007)
0807 04:22:14.008266 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:14.008274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:14.008608 (+ 334us) server_negotiation.cc:407] Connection header received
0807 04:22:14.008822 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:14.008827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:14.008895 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:14.009002 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:14.010334 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.011099 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.012451 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.012873 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.013011 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:14.016948 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:14.016953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:14.016957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:14.016988 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:14.019710 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.020324 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.020328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.020330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.020455 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.020820 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.020824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.020827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.021103 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:14.021321 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:14.021515 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 04:22:14.021685 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:14.068333 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:14.054634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.1.244.193:40007)
0807 04:22:14.054894 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:14.054900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:14.055134 (+ 234us) server_negotiation.cc:407] Connection header received
0807 04:22:14.055311 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:14.055315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:14.055379 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:14.055503 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:14.056882 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.057623 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.058849 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.059581 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.059718 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:14.063540 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:14.063546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:14.063549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:14.063589 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:14.066218 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.066778 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.066782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.066784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.066904 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.067187 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.067192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.067195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.067466 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:14.067648 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:14.067848 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 04:22:14.068074 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260807 04:22:14.070389 2699 tablet_service.cc:2908] Rejecting scan request for tablet 876ca89484924bb9ba4d0a232158eed9: Uninitialized: clean time has not yet been initialized
I20260807 04:22:14.073401 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2654
I20260807 04:22:14.085574 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:14.074287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.1.244.193:40007)
0807 04:22:14.074437 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:14.074441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:14.074827 (+ 386us) server_negotiation.cc:407] Connection header received
0807 04:22:14.074955 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:14.074959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:14.075033 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:14.075103 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:14.076256 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.076941 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.078239 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.078678 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.078780 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:14.080827 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:14.080831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:14.080834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:14.080861 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:14.083291 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.083874 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.083878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.083880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.084062 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.084365 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.084368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.084370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.084618 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:14.084889 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:14.085181 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 04:22:14.085362 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 04:22:14.086151 2780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:14.178179 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2654
I20260807 04:22:14.197774 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2791
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:14.214535 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:14.198622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.1.244.194:36759)
0807 04:22:14.198807 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:14.198811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:14.198851 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:22:14.199035 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:14.199039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:14.199100 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:14.199185 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:14.200817 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.201590 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.203380 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.203870 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.204123 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:14.208691 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:14.208704 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:14.208717 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:14.208757 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:14.211540 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.212427 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.212435 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.212437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.212590 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.212993 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.212997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.212999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.213308 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:14.213641 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:14.213958 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 04:22:14.214239 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:22:14.215193 2917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:14.304195 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2791
I20260807 04:22:14.323724 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2927
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[2547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:14.338390 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:14.324638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.1.244.195:43337)
0807 04:22:14.324840 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:14.324856 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:14.324918 (+ 62us) server_negotiation.cc:407] Connection header received
0807 04:22:14.325147 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:14.325151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:14.325213 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:14.325326 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:14.326740 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.327494 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.328865 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:14.329237 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:14.329329 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:14.333319 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:14.333333 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:14.333347 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:14.333387 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:14.336199 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.336790 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.336797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.336799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.336938 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:14.337248 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:14.337251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:14.337253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:14.337569 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:14.337777 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:14.337981 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 04:22:14.338145 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:22:14.339049 3053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:14.425431 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2927
I20260807 04:22:14.444103 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 2560
I20260807 04:22:14.445269 2623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:14.552693 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 2560
2026-08-07T04:22:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4136 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[3106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[3106](info): set up 2 sockets
Aug 07 04:22:14 dist-test-slave-zkpd krb5kdc[3106](info): commencing operation
krb5kdc: starting...
W20260807 04:22:15.646656 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:15 dist-test-slave-zkpd krb5kdc[3106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:15.836989 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36979
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:16.153970 3119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:16.154227 3119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:16.154313 3119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:16.161275 3119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:16.161368 3119 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 04:22:16.161406 3119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:16.161465 3119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:16.161513 3119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:16.161592 3119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:16.171038 3119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36979
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:16.172307 3119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:16.173563 3119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:16.185034 3124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:16.185110 3127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:16.185087 3125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:16.186097 3119 server_base.cc:1034] running on GCE node
I20260807 04:22:16.186882 3119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:16.188251 3119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:16.189503 3119 hybrid_clock.cc:648] HybridClock initialized: now 1786076536189448 us; error 31 us; skew 500 ppm
I20260807 04:22:16.189921 3119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:16 dist-test-slave-zkpd krb5kdc[3106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076536, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:16.193419 3119 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:16.194826 3119 webserver.cc:460] Webserver started at http://127.1.244.254:44699/ using document root <none> and password file <none>
I20260807 04:22:16.195482 3119 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:16.195567 3119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:16.195830 3119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:16.198693 3119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "3f74a195ba534607a6dc8406a337732e"
format_stamp: "Formatted at 2026-08-07 04:22:16 on dist-test-slave-zkpd"
server_key: "78517d8166fef2fc50beb0a6e3afdd2e"
server_key_iv: "ab44a617c59711380b2d99aa6892d428"
server_key_version: "encryptionkey@0"
I20260807 04:22:16.199433 3119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "3f74a195ba534607a6dc8406a337732e"
format_stamp: "Formatted at 2026-08-07 04:22:16 on dist-test-slave-zkpd"
server_key: "78517d8166fef2fc50beb0a6e3afdd2e"
server_key_iv: "ab44a617c59711380b2d99aa6892d428"
server_key_version: "encryptionkey@0"
I20260807 04:22:16.204422 3119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:22:16.207785 3134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:16.209084 3119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:16.209271 3119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "3f74a195ba534607a6dc8406a337732e"
format_stamp: "Formatted at 2026-08-07 04:22:16 on dist-test-slave-zkpd"
server_key: "78517d8166fef2fc50beb0a6e3afdd2e"
server_key_iv: "ab44a617c59711380b2d99aa6892d428"
server_key_version: "encryptionkey@0"
I20260807 04:22:16.209473 3119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:16.224706 3119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:16.228360 3119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:16.228623 3119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:16.249426 3119 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:36979
I20260807 04:22:16.249451 3196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:36979 every 8 connection(s)
I20260807 04:22:16.250810 3119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:16.252550 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3119
I20260807 04:22:16.252796 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:16.253651 2003 external_mini_cluster.cc:1428] Setting key 527b57ab4cd4d8d67a949a8cc985f704
I20260807 04:22:16.255672 3197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:22:16 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:16.269140 3197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: Bootstrap starting.
I20260807 04:22:16.272560 3197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:16.273566 3197 log.cc:824] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:16.276197 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:16.255698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.1.244.254:36979)
0807 04:22:16.256575 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:16.256587 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:16.256625 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:22:16.258202 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:16.258222 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:16.258593 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:16.258900 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:16.260700 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.262030 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.263727 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.264413 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.264541 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:16.269091 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:16.269098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:16.269110 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:16.269147 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:16.272225 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:16.272928 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:16.272939 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:16.272945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:16.273099 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:16.273468 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:16.273473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:16.273476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:16.274042 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:16.274355 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:16.274849 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 04:22:16.275229 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":439,"threads_started":1}
I20260807 04:22:16.276465 3197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: No bootstrap required, opened a new log
I20260807 04:22:16.283828 3197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f74a195ba534607a6dc8406a337732e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36979 } }
I20260807 04:22:16.284319 3197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:16.284401 3197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f74a195ba534607a6dc8406a337732e, State: Initialized, Role: FOLLOWER
I20260807 04:22:16.284878 3197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f74a195ba534607a6dc8406a337732e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36979 } }
I20260807 04:22:16.285075 3197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:16.285168 3197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:16.285286 3197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:16.287328 3197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f74a195ba534607a6dc8406a337732e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36979 } }
I20260807 04:22:16.287948 3197 leader_election.cc:302] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f74a195ba534607a6dc8406a337732e; no voters:
I20260807 04:22:16.288592 3197 leader_election.cc:288] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:16.288681 3202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:16.289409 3202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [term 1 LEADER]: Becoming Leader. State: Replica: 3f74a195ba534607a6dc8406a337732e, State: Running, Role: LEADER
I20260807 04:22:16.290077 3202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f74a195ba534607a6dc8406a337732e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36979 } }
I20260807 04:22:16.290756 3197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:16.293973 3204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f74a195ba534607a6dc8406a337732e. Latest consensus state: current_term: 1 leader_uuid: "3f74a195ba534607a6dc8406a337732e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f74a195ba534607a6dc8406a337732e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36979 } } }
I20260807 04:22:16.294411 3204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:16.295527 3209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:16.295506 3203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f74a195ba534607a6dc8406a337732e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f74a195ba534607a6dc8406a337732e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36979 } } }
I20260807 04:22:16.295950 3203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:16.300580 3209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:16.310087 3209 catalog_manager.cc:1347] Generated new cluster ID: 349efadad79b4f0892d49bc7e261d0a4
I20260807 04:22:16.310209 3209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:16.335295 3209 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:16.336856 3209 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:16.348343 3209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: Generated new TSK 0
I20260807 04:22:16.349300 3209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:16.415020 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:36979
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:16.751251 3223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:16.751514 3223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:16.751610 3223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:16.758610 3223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:16.758708 3223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:16.758786 3223 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:22:16.758855 3223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:16.767793 3223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:36979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:16.768973 3223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:16.770219 3223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:16.783537 3228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:16.783597 3229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:16.783576 3231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:16.784376 3223 server_base.cc:1034] running on GCE node
I20260807 04:22:16.784960 3223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:16.785772 3223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:16.787014 3223 hybrid_clock.cc:648] HybridClock initialized: now 1786076536786941 us; error 52 us; skew 500 ppm
I20260807 04:22:16.787307 3223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:16 dist-test-slave-zkpd krb5kdc[3106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076536, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:16.790532 3223 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:16.792249 3223 webserver.cc:460] Webserver started at http://127.1.244.193:37043/ using document root <none> and password file <none>
I20260807 04:22:16.792954 3223 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:16.793081 3223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:16.793427 3223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:16.796494 3223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "fd65f5a66103494e96016657f11afcfb"
format_stamp: "Formatted at 2026-08-07 04:22:16 on dist-test-slave-zkpd"
server_key: "014cfa40a510881223ddcdc3fc509c74"
server_key_iv: "3624de7bd702f5315659c7604c2af095"
server_key_version: "encryptionkey@0"
I20260807 04:22:16.797277 3223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "fd65f5a66103494e96016657f11afcfb"
format_stamp: "Formatted at 2026-08-07 04:22:16 on dist-test-slave-zkpd"
server_key: "014cfa40a510881223ddcdc3fc509c74"
server_key_iv: "3624de7bd702f5315659c7604c2af095"
server_key_version: "encryptionkey@0"
I20260807 04:22:16.802278 3223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:16.807288 3238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:16.809197 3223 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 04:22:16.809384 3223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "fd65f5a66103494e96016657f11afcfb"
format_stamp: "Formatted at 2026-08-07 04:22:16 on dist-test-slave-zkpd"
server_key: "014cfa40a510881223ddcdc3fc509c74"
server_key_iv: "3624de7bd702f5315659c7604c2af095"
server_key_version: "encryptionkey@0"
I20260807 04:22:16.809618 3223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:16.833662 3223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:16.838076 3223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:16.838356 3223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:16.841248 3223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:16.841359 3223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:16.841521 3223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:16.841610 3223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:22:16 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076536, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:16.863618 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:16.841624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:54297 (local address 127.1.244.254:36979)
0807 04:22:16.842580 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:16.842585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:16.843409 (+ 824us) server_negotiation.cc:407] Connection header received
0807 04:22:16.845436 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:16.845442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:16.845532 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:16.845648 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:16.847794 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.849193 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.851707 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.852305 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.852444 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:16.857550 (+ 5106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:16.857556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:16.857561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:16.857600 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:16.860179 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:16.861166 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:16.861172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:16.861175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:16.861339 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:16.861922 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:16.861927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:16.861932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:16.862218 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:16.862364 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:16.863025 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 04:22:16.863236 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":740,"threads_started":1}
I20260807 04:22:16.864986 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:16.842372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.1.244.193:54297)
0807 04:22:16.843248 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:16.843285 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:16.845136 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:16.845945 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:16.845959 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:16.846716 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:16.847527 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:16.847574 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.849400 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.849407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:16.851551 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:16.851563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.852572 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.852583 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:16.852929 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:16.853656 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:16.853731 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:16.857299 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:16.860329 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:16.860344 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:16.860361 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:16.860991 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:16.861505 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:16.861512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:16.861520 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:16.861788 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:16.862353 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:16.862360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:16.862679 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 04:22:16.862927 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 04:22:16.863414 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":382,"threads_started":1}
I20260807 04:22:16.885188 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:16.871090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:42885 (local address 127.1.244.254:36979)
0807 04:22:16.871262 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:16.871269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:16.872228 (+ 959us) server_negotiation.cc:407] Connection header received
0807 04:22:16.872371 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:16.872379 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:16.872489 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:16.872609 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:16.873415 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.874814 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.876120 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.876750 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.876886 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:16.878665 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:16.878673 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:16.878679 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:16.878723 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:16.882362 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:16.882958 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:16.882966 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:16.882971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:16.883138 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:16.883675 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:16.883680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:16.883684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:16.884056 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:16.884216 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:16.884781 (+ 565us) server_negotiation.cc:299] Negotiation successful
0807 04:22:16.884938 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:22:16.885572 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:16.871297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.1.244.193:42885)
0807 04:22:16.872086 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:16.872105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:16.872254 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:16.872620 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:16.872623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:16.873017 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:16.873283 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:16.873291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.874961 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.874964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:16.875984 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:16.875996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:16.876989 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:16.876997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:16.877157 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:16.877801 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:16.877828 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:16.878475 (+ 647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:16.882478 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:16.882490 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:16.882493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:16.882836 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:16.883268 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:16.883276 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:16.883281 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:16.883509 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:16.884203 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:16.884211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:16.884521 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 04:22:16.884699 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 04:22:16.885067 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":416,"threads_started":1}
I20260807 04:22:16.894892 3223 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:33469
I20260807 04:22:16.894918 3360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:33469 every 8 connection(s)
I20260807 04:22:16.896157 3223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:16.898123 3361 heartbeater.cc:347] Connected to a master server at 127.1.244.254:36979
I20260807 04:22:16.898335 3361 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:16.898955 3361 heartbeater.cc:511] Master 127.1.244.254:36979 requested a full tablet report, sending...
I20260807 04:22:16.900842 3151 ts_manager.cc:194] Registered new tserver with Master: fd65f5a66103494e96016657f11afcfb (127.1.244.193:33469)
I20260807 04:22:16.902266 3151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:54297
I20260807 04:22:16.903062 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3223
I20260807 04:22:16.903319 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:16.904052 2003 external_mini_cluster.cc:1428] Setting key 2b66d06a8f3aa23809f7e7e9d67ab65e
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:16.945024 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:36979
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:17.275342 3365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:17.275615 3365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:17.275712 3365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:17.282953 3365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:17.283070 3365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:17.283152 3365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:22:17.283234 3365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:17.293062 3365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:36979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:17.294241 3365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:17.295496 3365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:17.309000 3370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:17.308959 3373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:17.309167 3371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:17.309792 3365 server_base.cc:1034] running on GCE node
I20260807 04:22:17.310412 3365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:17.311336 3365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:17.312582 3365 hybrid_clock.cc:648] HybridClock initialized: now 1786076537312525 us; error 48 us; skew 500 ppm
I20260807 04:22:17.312915 3365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:17 dist-test-slave-zkpd krb5kdc[3106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:17.323498 3365 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:17.325099 3365 webserver.cc:460] Webserver started at http://127.1.244.194:46231/ using document root <none> and password file <none>
I20260807 04:22:17.325826 3365 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:17.325966 3365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:17.326297 3365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:17.329272 3365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "6a6261c9a0114192b4e91b0e41e8ca61"
format_stamp: "Formatted at 2026-08-07 04:22:17 on dist-test-slave-zkpd"
server_key: "d085335dc1bb957339c308bb0c15b4fe"
server_key_iv: "2bcf335a03bfb68f6288719e70384b3f"
server_key_version: "encryptionkey@0"
I20260807 04:22:17.330090 3365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "6a6261c9a0114192b4e91b0e41e8ca61"
format_stamp: "Formatted at 2026-08-07 04:22:17 on dist-test-slave-zkpd"
server_key: "d085335dc1bb957339c308bb0c15b4fe"
server_key_iv: "2bcf335a03bfb68f6288719e70384b3f"
server_key_version: "encryptionkey@0"
I20260807 04:22:17.335086 3365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 04:22:17.338428 3380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:17.339663 3365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:22:17.339838 3365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "6a6261c9a0114192b4e91b0e41e8ca61"
format_stamp: "Formatted at 2026-08-07 04:22:17 on dist-test-slave-zkpd"
server_key: "d085335dc1bb957339c308bb0c15b4fe"
server_key_iv: "2bcf335a03bfb68f6288719e70384b3f"
server_key_version: "encryptionkey@0"
I20260807 04:22:17.340142 3365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:17.358052 3365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:17.362303 3365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:17.362583 3365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:17.365239 3365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:17.365360 3365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:17.365504 3365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:17.365577 3365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:22:17 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:17.388897 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.365535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42535 (local address 127.1.244.254:36979)
0807 04:22:17.365746 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:17.365753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:17.367221 (+ 1468us) server_negotiation.cc:407] Connection header received
0807 04:22:17.369385 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:17.369398 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:17.369492 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:17.369603 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:17.371763 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.372897 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.375590 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.377228 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.377367 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.381960 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:17.381970 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:17.381978 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:17.382031 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:17.385618 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.386474 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.386479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.386483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.386631 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.387070 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.387076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.387079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.387357 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:17.387519 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:17.388284 (+ 765us) server_negotiation.cc:299] Negotiation successful
0807 04:22:17.388521 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:22:17.390494 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.366177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.1.244.194:42535)
0807 04:22:17.367023 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:17.367102 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:17.369103 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:17.369840 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:17.369851 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:17.370616 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:17.371497 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.371547 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.373068 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.373072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.375438 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.375451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.377492 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.377500 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.377709 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.378474 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:17.378501 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:17.381585 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:17.385793 (+ 4208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.385819 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.385834 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.386337 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.386750 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.386757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.386765 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.386954 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.387494 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:17.387501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:17.387837 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 04:22:17.388198 (+ 361us) client_negotiation.cc:240] Negotiation successful
0807 04:22:17.388777 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":356,"threads_started":1}
I20260807 04:22:17.410231 3365 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37971
I20260807 04:22:17.410300 3500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37971 every 8 connection(s)
I20260807 04:22:17.411360 3365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:17.412926 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3365
I20260807 04:22:17.413273 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:17.413906 3501 heartbeater.cc:347] Connected to a master server at 127.1.244.254:36979
I20260807 04:22:17.413990 2003 external_mini_cluster.cc:1428] Setting key faaf1977eb91bf5913e92291263f9ed4
I20260807 04:22:17.414143 3501 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:17.415303 3501 heartbeater.cc:511] Master 127.1.244.254:36979 requested a full tablet report, sending...
I20260807 04:22:17.417361 3151 ts_manager.cc:194] Registered new tserver with Master: 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971)
I20260807 04:22:17.418181 3151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:42535
I20260807 04:22:17.423349 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.398853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.1.244.194:60369)
0807 04:22:17.400603 (+ 1750us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:17.400628 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:17.400787 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:17.401126 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:17.401132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:17.401740 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:17.402223 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.402244 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.403987 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.403995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.406300 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.406320 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.407429 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.407438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.407670 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.408463 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:17.408490 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:17.409742 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:17.414862 (+ 5120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.414870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.414874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.415621 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.421735 (+ 6114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.421739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.421741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.421860 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.422583 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:17.422586 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:17.422727 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 04:22:17.422834 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:22:17.423055 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"thread_start_us":1241,"threads_started":1}
I20260807 04:22:17.423408 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.398726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:60369 (local address 127.1.244.254:36979)
0807 04:22:17.398943 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:17.398950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:17.400733 (+ 1783us) server_negotiation.cc:407] Connection header received
0807 04:22:17.400919 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:17.400922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:17.400984 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:17.401062 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:17.402457 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.403724 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.406475 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.407193 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.407290 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.410002 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:17.410012 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:17.410018 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:17.410064 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:17.413555 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.415801 (+ 2246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.415809 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.415815 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.416031 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.421996 (+ 5965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.422003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.422008 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.422464 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:17.422623 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:17.422990 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 04:22:17.423200 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:17.459398 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:36979
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:17.781581 3505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:17.781827 3505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:17.781917 3505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:17.788744 3505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:17.788842 3505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:17.788935 3505 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:22:17.789016 3505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:17.798384 3505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:36979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:17.799484 3505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:17.800745 3505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:17.813486 3510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:17.813568 3511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:17.813602 3513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:17.814667 3505 server_base.cc:1034] running on GCE node
I20260807 04:22:17.815342 3505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:17.816215 3505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:17.817413 3505 hybrid_clock.cc:648] HybridClock initialized: now 1786076537817344 us; error 51 us; skew 500 ppm
I20260807 04:22:17.817658 3505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:17 dist-test-slave-zkpd krb5kdc[3106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:17.820640 3505 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:17.821929 3505 webserver.cc:460] Webserver started at http://127.1.244.195:40483/ using document root <none> and password file <none>
I20260807 04:22:17.822557 3505 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:17.822679 3505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:17.822993 3505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:17.825855 3505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "a478384370f045dbb274e73d904f67af"
format_stamp: "Formatted at 2026-08-07 04:22:17 on dist-test-slave-zkpd"
server_key: "19c10ea1914ed9a41f57ff4f19235f6f"
server_key_iv: "01bcf6a3ee4b2756010139eb70599b92"
server_key_version: "encryptionkey@0"
I20260807 04:22:17.826644 3505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "a478384370f045dbb274e73d904f67af"
format_stamp: "Formatted at 2026-08-07 04:22:17 on dist-test-slave-zkpd"
server_key: "19c10ea1914ed9a41f57ff4f19235f6f"
server_key_iv: "01bcf6a3ee4b2756010139eb70599b92"
server_key_version: "encryptionkey@0"
I20260807 04:22:17.831482 3505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 04:22:17.834682 3520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:17.835839 3505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 04:22:17.836025 3505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "a478384370f045dbb274e73d904f67af"
format_stamp: "Formatted at 2026-08-07 04:22:17 on dist-test-slave-zkpd"
server_key: "19c10ea1914ed9a41f57ff4f19235f6f"
server_key_iv: "01bcf6a3ee4b2756010139eb70599b92"
server_key_version: "encryptionkey@0"
I20260807 04:22:17.836239 3505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:17.863854 3505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:17.868142 3505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:17.868436 3505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:17.870798 3505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:17.870908 3505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:17.871059 3505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:17.871133 3505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:22:17 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:17.899914 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.871583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59023 (local address 127.1.244.254:36979)
0807 04:22:17.871787 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:17.871794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:17.873164 (+ 1370us) server_negotiation.cc:407] Connection header received
0807 04:22:17.875385 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:17.875392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:17.875483 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:17.875594 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:17.878355 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.879338 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.882467 (+ 3129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.883217 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.883406 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.887670 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:17.887680 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:17.887688 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:17.887745 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:17.891145 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.892074 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.892079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.892085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.892263 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.893094 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.893099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.893103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.893395 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:17.893588 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:17.898978 (+ 5390us) server_negotiation.cc:299] Negotiation successful
0807 04:22:17.899347 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:22:17.900782 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.872136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.1.244.195:59023)
0807 04:22:17.872988 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:17.873039 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:17.875117 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:17.876062 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:17.876080 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:17.877028 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:17.878136 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.878164 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.879527 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.879534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.882240 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.882266 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.883496 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.883503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.883672 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.884383 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:17.884404 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:17.887375 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:17.891272 (+ 3897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.891284 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.891307 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.891943 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.892449 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.892457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.892463 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.892931 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.893707 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:17.893716 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:17.898459 (+ 4743us) client_negotiation.cc:769] Sending connection context
0807 04:22:17.898757 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 04:22:17.899267 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":381,"threads_started":1}
I20260807 04:22:17.905170 3361 heartbeater.cc:503] Master 127.1.244.254:36979 was elected leader, sending a full tablet report...
I20260807 04:22:17.921630 3505 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:38817
I20260807 04:22:17.921634 3643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:38817 every 8 connection(s)
I20260807 04:22:17.923337 3505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:17.925573 3644 heartbeater.cc:347] Connected to a master server at 127.1.244.254:36979
I20260807 04:22:17.925606 3607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.908870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.1.244.195:38021)
0807 04:22:17.909556 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:17.909572 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:17.909706 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:17.910203 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:17.910211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:17.910962 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:17.911560 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.911579 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.913176 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.913181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.914872 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:17.914902 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.916091 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.916101 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:17.916362 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.917182 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:17.917211 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:17.918576 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:17.922702 (+ 4126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.922706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.922708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.923139 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.923574 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:17.923578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:17.923580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:17.923694 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:17.924440 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:17.924444 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:17.924641 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 04:22:17.924802 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 04:22:17.925124 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":408,"threads_started":1}
I20260807 04:22:17.925992 3644 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:17.925992 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.908661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:38021 (local address 127.1.244.254:36979)
0807 04:22:17.908834 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:17.908842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:17.909647 (+ 805us) server_negotiation.cc:407] Connection header received
0807 04:22:17.909836 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:17.909843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:17.909927 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:17.910027 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:17.911759 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.913032 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.915073 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.915848 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.916008 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.918810 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:17.918820 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:17.918827 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:17.918867 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:17.922582 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.923277 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.923286 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.923291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.923464 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.923808 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.923814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.923820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.924261 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:17.925231 (+ 970us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:17.925353 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 04:22:17.925594 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 04:22:17.926620 3644 heartbeater.cc:511] Master 127.1.244.254:36979 requested a full tablet report, sending...
I20260807 04:22:17.928004 3151 ts_manager.cc:194] Registered new tserver with Master: a478384370f045dbb274e73d904f67af (127.1.244.195:38817)
I20260807 04:22:17.928228 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3505
I20260807 04:22:17.928435 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:17.928987 2003 external_mini_cluster.cc:1428] Setting key 33eb248bbb64f38e357dd56533097545
I20260807 04:22:17.929160 3151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:59023
I20260807 04:22:17.932022 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:22:17 dist-test-slave-zkpd krb5kdc[3106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:22:17 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:17.978971 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:17.966107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.1.244.254:36979)
0807 04:22:17.966294 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:17.966299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:17.966590 (+ 291us) server_negotiation.cc:407] Connection header received
0807 04:22:17.966740 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:17.966744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:17.966805 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:17.966896 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:17.968387 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.969127 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.970298 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:17.970682 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:17.970768 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:17.974312 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:17.974317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:17.974321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:17.974353 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:17.976946 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.977464 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.977468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.977470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.977580 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:17.977872 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:17.977875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:17.977877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:17.978135 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:17.978310 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:17.978529 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 04:22:17.978688 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:22:17.982841 3151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:17.989624 3151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:18.028553 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.001953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52482 (local address 127.1.244.195:38817)
0807 04:22:18.020216 (+ 18263us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.020225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.020261 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:22:18.020362 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.020374 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.020669 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.021357 (+ 688us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.021975 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.023754 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.026177 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.027367 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.027512 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.027751 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.028034 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.028194 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18063,"thread_start_us":17811,"threads_started":1}
I20260807 04:22:18.028633 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.001742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38817 (local address 127.0.0.1:52482)
0807 04:22:18.016908 (+ 15166us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.016980 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.017212 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.020923 (+ 3711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.020932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.020966 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.021731 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.021748 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.023947 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.023952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.025972 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.025987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.027563 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.027570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.027748 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.027816 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.027992 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.028182 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14836,"mutex_wait_us":113,"thread_start_us":15700,"threads_started":1}
I20260807 04:22:18.029474 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.003193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43842 (local address 127.1.244.194:37971)
0807 04:22:18.018613 (+ 15420us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.018626 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.020602 (+ 1976us) server_negotiation.cc:407] Connection header received
0807 04:22:18.020760 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.020768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.020996 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.021222 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.022439 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.023935 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.026691 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.028091 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.028208 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.028409 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.029035 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.029208 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15223,"thread_start_us":15079,"threads_started":1}
I20260807 04:22:18.029487 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.002417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37971 (local address 127.0.0.1:43842)
0807 04:22:18.020452 (+ 18035us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.020477 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.020611 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.021233 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.021238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.021268 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.022236 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.022252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.024049 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.024060 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.026505 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.026520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.028636 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.028643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.028836 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.028866 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.028963 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.029086 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17900,"mutex_wait_us":78,"thread_start_us":17719,"threads_started":1}
I20260807 04:22:18.030144 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.001958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33469 (local address 127.0.0.1:44248)
0807 04:22:18.019279 (+ 17321us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.019308 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.019472 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.021258 (+ 1786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.021262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.021280 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.021858 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.021898 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.023828 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.023833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.026504 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.026520 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.028599 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.028611 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.029153 (+ 542us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.029199 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.029299 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.029425 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17050,"mutex_wait_us":217,"thread_start_us":16883,"threads_started":1}
I20260807 04:22:18.030959 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.002602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.1.244.193:33469)
0807 04:22:18.020642 (+ 18040us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.020652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.020684 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:22:18.020809 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.020817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.021045 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.021655 (+ 610us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.022063 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.023694 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.026691 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.028091 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.028225 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.028429 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.030500 (+ 2071us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.030666 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17876,"thread_start_us":18096,"threads_started":1}
I20260807 04:22:18.032863 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 95b1a307b69f4dcc86520b885fd0a8d8 (DEFAULT_TABLE table=test-table [id=c8cf839c25794efeb6e90d36a538455e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:18.032809 3571 tablet_service.cc:1467] Processing CreateTablet for tablet 95b1a307b69f4dcc86520b885fd0a8d8 (DEFAULT_TABLE table=test-table [id=c8cf839c25794efeb6e90d36a538455e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:18.034073 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b1a307b69f4dcc86520b885fd0a8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.034250 3571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b1a307b69f4dcc86520b885fd0a8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.034353 3289 tablet_service.cc:1467] Processing CreateTablet for tablet 95b1a307b69f4dcc86520b885fd0a8d8 (DEFAULT_TABLE table=test-table [id=c8cf839c25794efeb6e90d36a538455e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:18.035589 3289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b1a307b69f4dcc86520b885fd0a8d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.045428 3664 tablet_bootstrap.cc:492] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Bootstrap starting.
I20260807 04:22:18.048352 3664 tablet_bootstrap.cc:654] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.049510 3666 tablet_bootstrap.cc:492] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Bootstrap starting.
I20260807 04:22:18.049809 3664 log.cc:824] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:18.052760 3666 tablet_bootstrap.cc:654] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.053560 3664 tablet_bootstrap.cc:492] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: No bootstrap required, opened a new log
I20260807 04:22:18.053851 3664 ts_tablet_manager.cc:1397] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:22:18.054039 3666 log.cc:824] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:18.055151 3665 tablet_bootstrap.cc:492] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Bootstrap starting.
I20260807 04:22:18.058182 3666 tablet_bootstrap.cc:492] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: No bootstrap required, opened a new log
I20260807 04:22:18.058432 3666 ts_tablet_manager.cc:1397] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 04:22:18.058599 3665 tablet_bootstrap.cc:654] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.059764 3665 log.cc:824] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:18.062566 3665 tablet_bootstrap.cc:492] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: No bootstrap required, opened a new log
I20260807 04:22:18.062808 3665 ts_tablet_manager.cc:1397] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:22:18.064316 3664 raft_consensus.cc:348] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.064736 3664 raft_consensus.cc:374] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.064843 3664 raft_consensus.cc:729] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6261c9a0114192b4e91b0e41e8ca61, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.065433 3664 consensus_queue.cc:260] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.066248 3666 raft_consensus.cc:348] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.066569 3666 raft_consensus.cc:374] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.066668 3666 raft_consensus.cc:729] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd65f5a66103494e96016657f11afcfb, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.067394 3666 consensus_queue.cc:260] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.072309 3666 ts_tablet_manager.cc:1428] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260807 04:22:18.073144 3501 heartbeater.cc:503] Master 127.1.244.254:36979 was elected leader, sending a full tablet report...
I20260807 04:22:18.074877 3664 ts_tablet_manager.cc:1428] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260807 04:22:18.074993 3665 raft_consensus.cc:348] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.075309 3665 raft_consensus.cc:374] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.075373 3665 raft_consensus.cc:729] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a478384370f045dbb274e73d904f67af, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.075846 3665 consensus_queue.cc:260] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.077688 3665 ts_tablet_manager.cc:1428] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 04:22:18.077916 3644 heartbeater.cc:503] Master 127.1.244.254:36979 was elected leader, sending a full tablet report...
I20260807 04:22:18.106839 3671 raft_consensus.cc:482] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:18.107136 3671 raft_consensus.cc:504] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.108825 3671 leader_election.cc:288] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a478384370f045dbb274e73d904f67af (127.1.244.195:38817), 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971)
I20260807 04:22:18.116047 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.109218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:55923 (local address 127.1.244.195:38817)
0807 04:22:18.109426 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.109434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.110197 (+ 763us) server_negotiation.cc:407] Connection header received
0807 04:22:18.110381 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.110389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.110481 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.110545 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.111161 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.111962 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.114004 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.115045 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.115227 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.115301 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.115640 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.115766 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:22:18.116191 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.109162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38817 (local address 127.1.244.193:55923)
0807 04:22:18.110069 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.110089 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.110222 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.110593 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.110600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.110629 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.111015 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.111024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.112082 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.112090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.113859 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.113871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.115255 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.115261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.115452 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.115490 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.115596 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.115757 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":23,"thread_start_us":496,"threads_started":1}
I20260807 04:22:18.116300 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.109250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:36557 (local address 127.1.244.194:37971)
0807 04:22:18.109471 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.109479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.110296 (+ 817us) server_negotiation.cc:407] Connection header received
0807 04:22:18.110464 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.110468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.110541 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.110631 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.111270 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.112052 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.114182 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.115112 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.115269 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.115340 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.115863 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.116017 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:22:18.116974 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.109317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37971 (local address 127.1.244.193:36557)
0807 04:22:18.110178 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.110196 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.110342 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.110664 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.110672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.110705 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.111113 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.111126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.112177 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.112181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.114045 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.114060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.115497 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.115502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.115684 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.115718 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.116469 (+ 751us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.116601 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":3,"thread_start_us":540,"threads_started":1}
I20260807 04:22:18.117200 3593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b1a307b69f4dcc86520b885fd0a8d8" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a478384370f045dbb274e73d904f67af" is_pre_election: true
I20260807 04:22:18.117723 3593 raft_consensus.cc:2454] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 0.
I20260807 04:22:18.117820 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b1a307b69f4dcc86520b885fd0a8d8" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" is_pre_election: true
I20260807 04:22:18.118314 3450 raft_consensus.cc:2454] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 0.
I20260807 04:22:18.118387 3239 leader_election.cc:302] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a478384370f045dbb274e73d904f67af, fd65f5a66103494e96016657f11afcfb; no voters:
I20260807 04:22:18.118773 3671 raft_consensus.cc:2781] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:18.118922 3671 raft_consensus.cc:482] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:18.119050 3671 raft_consensus.cc:3037] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.121309 3671 raft_consensus.cc:504] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.121953 3671 leader_election.cc:288] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 election: Requested vote from peers a478384370f045dbb274e73d904f67af (127.1.244.195:38817), 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971)
I20260807 04:22:18.122272 3593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b1a307b69f4dcc86520b885fd0a8d8" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a478384370f045dbb274e73d904f67af"
I20260807 04:22:18.122403 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b1a307b69f4dcc86520b885fd0a8d8" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6261c9a0114192b4e91b0e41e8ca61"
I20260807 04:22:18.122485 3593 raft_consensus.cc:3037] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.122604 3450 raft_consensus.cc:3037] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.124786 3593 raft_consensus.cc:2454] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 1.
I20260807 04:22:18.124835 3450 raft_consensus.cc:2454] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 1.
I20260807 04:22:18.125174 3239 leader_election.cc:302] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a478384370f045dbb274e73d904f67af, fd65f5a66103494e96016657f11afcfb; no voters:
I20260807 04:22:18.125483 3671 raft_consensus.cc:2781] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:18.125937 3671 raft_consensus.cc:686] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 1 LEADER]: Becoming Leader. State: Replica: fd65f5a66103494e96016657f11afcfb, State: Running, Role: LEADER
I20260807 04:22:18.126381 3671 consensus_queue.cc:237] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } }
I20260807 04:22:18.130506 3151 catalog_manager.cc:5495] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to fd65f5a66103494e96016657f11afcfb (127.1.244.193). New cstate: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } health_report { overall_health: UNKNOWN } } }
W20260807 04:22:18.148581 3362 tablet.cc:2369] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:22:18.159868 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.153404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.1.244.193:33469)
0807 04:22:18.153566 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.153570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.153792 (+ 222us) server_negotiation.cc:407] Connection header received
0807 04:22:18.154085 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.154089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.154169 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.154285 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:18.154740 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.155715 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.157736 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.158130 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.158264 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.158773 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.159280 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.159434 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.159530 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.159619 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 04:22:18.167222 3502 tablet.cc:2369] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:22:18.167670 3450 raft_consensus.cc:1261] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Refusing update from remote peer fd65f5a66103494e96016657f11afcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:18.167675 3593 raft_consensus.cc:1261] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Refusing update from remote peer fd65f5a66103494e96016657f11afcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:18.168630 3675 consensus_queue.cc:1035] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:18.168992 3671 consensus_queue.cc:1035] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:22:18.175297 3645 tablet.cc:2369] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:22:18.184305 3680 mvcc.cc:205] Tried to move back new op lower bound from 7315769500325351424 to 7315769500169019392. Current Snapshot: MvccSnapshot[applied={T|T < 7315769500325351424 or (T in {7315769500325351424})}]
I20260807 04:22:18.199453 3151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:18.199699 3151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:18.203297 3151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: Sending DeleteTablet for 3 replicas of tablet 95b1a307b69f4dcc86520b885fd0a8d8
I20260807 04:22:18.204586 3571 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b1a307b69f4dcc86520b885fd0a8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:18 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:52482
I20260807 04:22:18.204864 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b1a307b69f4dcc86520b885fd0a8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:18 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:43842
I20260807 04:22:18.204586 3289 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b1a307b69f4dcc86520b885fd0a8d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:18 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44248
I20260807 04:22:18.212703 3151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:18.214277 3151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:18.216229 3689 tablet_replica.cc:330] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: stopping tablet replica
I20260807 04:22:18.217310 3689 raft_consensus.cc:2229] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:18.218820 3689 raft_consensus.cc:2258] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:18.219846 3688 tablet_replica.cc:330] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: stopping tablet replica
I20260807 04:22:18.220320 3687 tablet_replica.cc:330] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: stopping tablet replica
I20260807 04:22:18.220701 3688 raft_consensus.cc:2229] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:18.220714 3687 raft_consensus.cc:2229] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:18.221199 3687 raft_consensus.cc:2258] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:18.221835 3688 raft_consensus.cc:2258] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:18.223387 3689 ts_tablet_manager.cc:1905] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:18.223387 3687 ts_tablet_manager.cc:1905] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:18.224128 3688 ts_tablet_manager.cc:1905] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:18.228294 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2fb46ea31d401494440e3d554195c5 (DEFAULT_TABLE table=test-table [id=e860b813d7d748d099204f555d624b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:18.226862 3289 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2fb46ea31d401494440e3d554195c5 (DEFAULT_TABLE table=test-table [id=e860b813d7d748d099204f555d624b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:18.228933 3289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2fb46ea31d401494440e3d554195c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.230422 3687 ts_tablet_manager.cc:1918] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:18.230587 3687 log.cc:1196] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/95b1a307b69f4dcc86520b885fd0a8d8
I20260807 04:22:18.231161 3687 ts_tablet_manager.cc:1939] T 95b1a307b69f4dcc86520b885fd0a8d8 P a478384370f045dbb274e73d904f67af: Deleting consensus metadata
I20260807 04:22:18.231696 3688 ts_tablet_manager.cc:1918] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:18.231978 3688 log.cc:1196] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/95b1a307b69f4dcc86520b885fd0a8d8
I20260807 04:22:18.232529 3688 ts_tablet_manager.cc:1939] T 95b1a307b69f4dcc86520b885fd0a8d8 P 6a6261c9a0114192b4e91b0e41e8ca61: Deleting consensus metadata
I20260807 04:22:18.233302 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2fb46ea31d401494440e3d554195c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.233361 3135 catalog_manager.cc:4841] TS a478384370f045dbb274e73d904f67af (127.1.244.195:38817): tablet 95b1a307b69f4dcc86520b885fd0a8d8 (table test-table [id=c8cf839c25794efeb6e90d36a538455e]) successfully deleted
I20260807 04:22:18.237373 3666 tablet_bootstrap.cc:492] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Bootstrap starting.
I20260807 04:22:18.239022 3571 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2fb46ea31d401494440e3d554195c5 (DEFAULT_TABLE table=test-table [id=e860b813d7d748d099204f555d624b87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:18.239478 3571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2fb46ea31d401494440e3d554195c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.239574 3666 tablet_bootstrap.cc:654] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.241317 3689 ts_tablet_manager.cc:1918] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:18.241528 3689 log.cc:1196] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/95b1a307b69f4dcc86520b885fd0a8d8
I20260807 04:22:18.242143 3689 ts_tablet_manager.cc:1939] T 95b1a307b69f4dcc86520b885fd0a8d8 P fd65f5a66103494e96016657f11afcfb: Deleting consensus metadata
I20260807 04:22:18.244346 3137 catalog_manager.cc:4841] TS fd65f5a66103494e96016657f11afcfb (127.1.244.193:33469): tablet 95b1a307b69f4dcc86520b885fd0a8d8 (table test-table [id=c8cf839c25794efeb6e90d36a538455e]) successfully deleted
I20260807 04:22:18.244797 3136 catalog_manager.cc:4841] TS 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971): tablet 95b1a307b69f4dcc86520b885fd0a8d8 (table test-table [id=c8cf839c25794efeb6e90d36a538455e]) successfully deleted
I20260807 04:22:18.248934 3665 tablet_bootstrap.cc:492] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Bootstrap starting.
I20260807 04:22:18.249557 3664 tablet_bootstrap.cc:492] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Bootstrap starting.
I20260807 04:22:18.250524 3665 tablet_bootstrap.cc:654] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.251061 3664 tablet_bootstrap.cc:654] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.257306 3666 tablet_bootstrap.cc:492] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: No bootstrap required, opened a new log
I20260807 04:22:18.257485 3666 ts_tablet_manager.cc:1397] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.001s
I20260807 04:22:18.258296 3666 raft_consensus.cc:348] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.259291 3666 raft_consensus.cc:374] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.259523 3666 raft_consensus.cc:729] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd65f5a66103494e96016657f11afcfb, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.260032 3666 consensus_queue.cc:260] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.264405 3666 ts_tablet_manager.cc:1428] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260807 04:22:18.265556 3664 tablet_bootstrap.cc:492] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: No bootstrap required, opened a new log
I20260807 04:22:18.265693 3664 ts_tablet_manager.cc:1397] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
I20260807 04:22:18.266475 3664 raft_consensus.cc:348] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.266692 3664 raft_consensus.cc:374] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.266794 3664 raft_consensus.cc:729] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6261c9a0114192b4e91b0e41e8ca61, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.267079 3664 consensus_queue.cc:260] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.267438 3665 tablet_bootstrap.cc:492] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: No bootstrap required, opened a new log
I20260807 04:22:18.267588 3665 ts_tablet_manager.cc:1397] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.012s
I20260807 04:22:18.268337 3665 raft_consensus.cc:348] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.268604 3665 raft_consensus.cc:374] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.268713 3665 raft_consensus.cc:729] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a478384370f045dbb274e73d904f67af, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.269006 3665 consensus_queue.cc:260] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.269589 3665 ts_tablet_manager.cc:1428] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 04:22:18.270438 3664 ts_tablet_manager.cc:1428] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 04:22:18.439777 3670 raft_consensus.cc:482] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:18.440052 3670 raft_consensus.cc:504] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.441637 3670 leader_election.cc:288] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd65f5a66103494e96016657f11afcfb (127.1.244.193:33469), a478384370f045dbb274e73d904f67af (127.1.244.195:38817)
I20260807 04:22:18.449630 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.442223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55055 (local address 127.1.244.195:38817)
0807 04:22:18.442431 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.442439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.442988 (+ 549us) server_negotiation.cc:407] Connection header received
0807 04:22:18.443158 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.443164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.443272 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.443383 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.444260 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.445477 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.448025 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.448723 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.448884 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.448969 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.449248 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.449372 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:22:18.449870 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.442074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38817 (local address 127.1.244.194:55055)
0807 04:22:18.442911 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.442925 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.443021 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.443438 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.443442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.443472 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.444029 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.444051 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.445606 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.445613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.447796 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.447812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.448932 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.448938 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.449096 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.449119 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.449221 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.449343 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":252,"thread_start_us":365,"threads_started":1}
I20260807 04:22:18.450601 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.442080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42449 (local address 127.1.244.193:33469)
0807 04:22:18.442263 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.442270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.442906 (+ 636us) server_negotiation.cc:407] Connection header received
0807 04:22:18.443151 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.443157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.443243 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.443342 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:18.444342 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.445264 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.447978 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.449302 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.449466 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.449542 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.450123 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.450264 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:22:18.450570 3593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2fb46ea31d401494440e3d554195c5" candidate_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a478384370f045dbb274e73d904f67af" is_pre_election: true
I20260807 04:22:18.450810 3593 raft_consensus.cc:2454] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6261c9a0114192b4e91b0e41e8ca61 in term 0.
I20260807 04:22:18.451349 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.442061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33469 (local address 127.1.244.194:42449)
0807 04:22:18.442787 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.442808 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.442965 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.443553 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.443559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.443596 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:18.444157 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.444170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.445413 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.445418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.447804 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.447812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.449714 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.449722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.449908 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.449950 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.450797 (+ 847us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.450935 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":111,"thread_start_us":341,"threads_started":1}
I20260807 04:22:18.451308 3381 leader_election.cc:302] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6261c9a0114192b4e91b0e41e8ca61, a478384370f045dbb274e73d904f67af; no voters:
I20260807 04:22:18.451701 3670 raft_consensus.cc:2781] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:18.451829 3670 raft_consensus.cc:482] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:18.451956 3670 raft_consensus.cc:3037] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.452497 3316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2fb46ea31d401494440e3d554195c5" candidate_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd65f5a66103494e96016657f11afcfb" is_pre_election: true
I20260807 04:22:18.452930 3316 raft_consensus.cc:2454] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6261c9a0114192b4e91b0e41e8ca61 in term 0.
I20260807 04:22:18.455209 3670 raft_consensus.cc:504] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.455924 3670 leader_election.cc:288] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [CANDIDATE]: Term 1 election: Requested vote from peers fd65f5a66103494e96016657f11afcfb (127.1.244.193:33469), a478384370f045dbb274e73d904f67af (127.1.244.195:38817)
I20260807 04:22:18.456349 3593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2fb46ea31d401494440e3d554195c5" candidate_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a478384370f045dbb274e73d904f67af"
I20260807 04:22:18.456331 3316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2fb46ea31d401494440e3d554195c5" candidate_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd65f5a66103494e96016657f11afcfb"
I20260807 04:22:18.456547 3593 raft_consensus.cc:3037] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.456562 3316 raft_consensus.cc:3037] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.458590 3593 raft_consensus.cc:2454] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6261c9a0114192b4e91b0e41e8ca61 in term 1.
I20260807 04:22:18.458962 3381 leader_election.cc:302] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6261c9a0114192b4e91b0e41e8ca61, a478384370f045dbb274e73d904f67af; no voters:
I20260807 04:22:18.459260 3670 raft_consensus.cc:2781] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:18.459770 3670 raft_consensus.cc:686] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 LEADER]: Becoming Leader. State: Replica: 6a6261c9a0114192b4e91b0e41e8ca61, State: Running, Role: LEADER
I20260807 04:22:18.459872 3316 raft_consensus.cc:2454] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6261c9a0114192b4e91b0e41e8ca61 in term 1.
I20260807 04:22:18.460487 3670 consensus_queue.cc:237] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.464166 3151 catalog_manager.cc:5495] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } health_report { overall_health: UNKNOWN } } }
Aug 07 04:22:18 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076536, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:18.518589 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.502217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36979 (local address 127.0.0.1:36138)
0807 04:22:18.503257 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.503279 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.503440 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.503792 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.503801 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.504884 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:18.505278 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.505293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.506469 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.506475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.507989 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.508008 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.508714 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.508723 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.508926 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.509665 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:18.509691 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:18.512688 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:18.516078 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:18.516092 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:18.516100 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:18.516541 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:18.516876 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:18.516882 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:18.516886 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:18.517023 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:18.517504 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:18.517512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:18.517682 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.517844 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.518123 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":564,"threads_started":1}
I20260807 04:22:18.518652 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.502203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.1.244.254:36979)
0807 04:22:18.503012 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.503020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.503368 (+ 348us) server_negotiation.cc:407] Connection header received
0807 04:22:18.503556 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.503560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.503616 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.503707 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:18.505462 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.506336 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.508126 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.508507 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.508595 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.512952 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:18.512961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:18.512967 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:18.513008 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:18.515939 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.516655 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.516660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.516664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.516778 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.517130 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.517134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.517137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.517386 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:18.517528 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.517941 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.518277 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":515,"thread_start_us":331,"threads_started":1}
I20260807 04:22:18.524000 3151 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:36138:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 04:22:18.525137 3151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:18.532398 3430 tablet_service.cc:1467] Processing CreateTablet for tablet ffb4fdb6017a436b9d6e275d505e35b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=827e233a6a524dcc8d0e56c17cc241f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:22:18.532398 3289 tablet_service.cc:1467] Processing CreateTablet for tablet ffb4fdb6017a436b9d6e275d505e35b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=827e233a6a524dcc8d0e56c17cc241f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:22:18.532720 3571 tablet_service.cc:1467] Processing CreateTablet for tablet ffb4fdb6017a436b9d6e275d505e35b9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=827e233a6a524dcc8d0e56c17cc241f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:22:18.532976 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb4fdb6017a436b9d6e275d505e35b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.533003 3289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb4fdb6017a436b9d6e275d505e35b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.533310 3571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb4fdb6017a436b9d6e275d505e35b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:18.540892 3664 tablet_bootstrap.cc:492] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: Bootstrap starting.
I20260807 04:22:18.541105 3665 tablet_bootstrap.cc:492] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: Bootstrap starting.
I20260807 04:22:18.541117 3666 tablet_bootstrap.cc:492] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: Bootstrap starting.
I20260807 04:22:18.542641 3664 tablet_bootstrap.cc:654] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.542876 3666 tablet_bootstrap.cc:654] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.543057 3665 tablet_bootstrap.cc:654] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:18.552511 3665 tablet_bootstrap.cc:492] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: No bootstrap required, opened a new log
I20260807 04:22:18.552695 3665 ts_tablet_manager.cc:1397] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20260807 04:22:18.553162 3666 tablet_bootstrap.cc:492] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: No bootstrap required, opened a new log
I20260807 04:22:18.553366 3666 ts_tablet_manager.cc:1397] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260807 04:22:18.553483 3664 tablet_bootstrap.cc:492] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: No bootstrap required, opened a new log
I20260807 04:22:18.553498 3665 raft_consensus.cc:348] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.553758 3664 ts_tablet_manager.cc:1397] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260807 04:22:18.553782 3665 raft_consensus.cc:374] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.553874 3665 raft_consensus.cc:729] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a478384370f045dbb274e73d904f67af, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.554158 3665 consensus_queue.cc:260] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.554375 3666 raft_consensus.cc:348] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.554684 3666 raft_consensus.cc:374] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.554773 3666 raft_consensus.cc:729] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd65f5a66103494e96016657f11afcfb, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.554749 3672 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.555120 3672 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.555053 3666 consensus_queue.cc:260] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.555501 3675 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.555789 3675 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.554749 3665 ts_tablet_manager.cc:1428] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 04:22:18.556531 3664 raft_consensus.cc:348] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.556845 3664 raft_consensus.cc:374] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:18.556972 3664 raft_consensus.cc:729] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6261c9a0114192b4e91b0e41e8ca61, State: Initialized, Role: FOLLOWER
I20260807 04:22:18.557299 3664 consensus_queue.cc:260] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.557806 3670 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.557996 3664 ts_tablet_manager.cc:1428] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260807 04:22:18.558208 3670 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.558892 3666 ts_tablet_manager.cc:1428] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260807 04:22:18.580278 3675 raft_consensus.cc:482] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:18.580556 3675 raft_consensus.cc:504] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.581413 3675 leader_election.cc:288] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971), a478384370f045dbb274e73d904f67af (127.1.244.195:38817)
I20260807 04:22:18.581897 3593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb4fdb6017a436b9d6e275d505e35b9" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a478384370f045dbb274e73d904f67af" is_pre_election: true
I20260807 04:22:18.581959 3449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb4fdb6017a436b9d6e275d505e35b9" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" is_pre_election: true
I20260807 04:22:18.582165 3593 raft_consensus.cc:2454] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 0.
I20260807 04:22:18.582310 3449 raft_consensus.cc:2454] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 0.
I20260807 04:22:18.582679 3239 leader_election.cc:302] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a478384370f045dbb274e73d904f67af, fd65f5a66103494e96016657f11afcfb; no voters:
I20260807 04:22:18.583053 3675 raft_consensus.cc:2781] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:18.583184 3675 raft_consensus.cc:482] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:18.583395 3675 raft_consensus.cc:3037] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.587100 3675 raft_consensus.cc:504] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.587855 3675 leader_election.cc:288] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 election: Requested vote from peers 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971), a478384370f045dbb274e73d904f67af (127.1.244.195:38817)
I20260807 04:22:18.588332 3449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb4fdb6017a436b9d6e275d505e35b9" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6261c9a0114192b4e91b0e41e8ca61"
I20260807 04:22:18.588346 3593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb4fdb6017a436b9d6e275d505e35b9" candidate_uuid: "fd65f5a66103494e96016657f11afcfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a478384370f045dbb274e73d904f67af"
I20260807 04:22:18.588557 3449 raft_consensus.cc:3037] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.588619 3593 raft_consensus.cc:3037] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:18.590771 3449 raft_consensus.cc:2454] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 1.
I20260807 04:22:18.591295 3240 leader_election.cc:302] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6261c9a0114192b4e91b0e41e8ca61, fd65f5a66103494e96016657f11afcfb; no voters:
I20260807 04:22:18.591652 3675 raft_consensus.cc:2781] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:18.591853 3593 raft_consensus.cc:2454] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd65f5a66103494e96016657f11afcfb in term 1.
I20260807 04:22:18.592481 3675 raft_consensus.cc:686] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [term 1 LEADER]: Becoming Leader. State: Replica: fd65f5a66103494e96016657f11afcfb, State: Running, Role: LEADER
I20260807 04:22:18.592834 3675 consensus_queue.cc:237] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } }
I20260807 04:22:18.594964 3671 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: TxnStatusTablet state changed. Reason: New leader fd65f5a66103494e96016657f11afcfb. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.595300 3671 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:22:18.596750 3151 catalog_manager.cc:5495] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to fd65f5a66103494e96016657f11afcfb (127.1.244.193). New cstate: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } health_report { overall_health: UNKNOWN } } }
I20260807 04:22:18.597266 3709 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 04:22:18.597489 3709 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 04:22:18.618440 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.612228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33469 (local address 127.0.0.1:44258)
0807 04:22:18.612453 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.612471 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.612611 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.613112 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.613115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.613125 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:18.613323 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.613331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.615020 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.615024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.616074 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.616086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.617313 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.617320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.617505 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.617565 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.618006 (+ 441us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.618021 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.618080 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.618211 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 04:22:18.618554 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.612296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.1.244.193:33469)
0807 04:22:18.612544 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.612551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.612575 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:22:18.612829 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.612835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.612970 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.613090 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:18.613478 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.614878 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.616223 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.617109 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.617269 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.617753 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.617902 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.618010 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.618118 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.618233 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 04:22:18.622115 3449 raft_consensus.cc:1261] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Refusing update from remote peer fd65f5a66103494e96016657f11afcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:18.622133 3593 raft_consensus.cc:1261] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Refusing update from remote peer fd65f5a66103494e96016657f11afcfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:18.622746 3671 consensus_queue.cc:1035] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:18.623090 3675 consensus_queue.cc:1035] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:18.624969 3672 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: TxnStatusTablet state changed. Reason: New leader fd65f5a66103494e96016657f11afcfb. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.625264 3672 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.627141 3675 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.627419 3675 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:22:18.627604 3678 mvcc.cc:205] Tried to move back new op lower bound from 7315769502189875200 to 7315769502078623744. Current Snapshot: MvccSnapshot[applied={T|T < 7315769502189875200 or (T in {7315769502189875200})}]
I20260807 04:22:18.629446 3672 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.629701 3672 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P a478384370f045dbb274e73d904f67af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.636838 3670 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: TxnStatusTablet state changed. Reason: New leader fd65f5a66103494e96016657f11afcfb. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.637207 3670 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.638923 3671 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.639240 3671 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P fd65f5a66103494e96016657f11afcfb: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:22:18.641008 3670 tablet_replica.cc:439] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fd65f5a66103494e96016657f11afcfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 } } peers { permanent_uuid: "6a6261c9a0114192b4e91b0e41e8ca61" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 37971 } } peers { permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 } } }
I20260807 04:22:18.641301 3670 tablet_replica.cc:442] T ffb4fdb6017a436b9d6e275d505e35b9 P 6a6261c9a0114192b4e91b0e41e8ca61: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:18.644301 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.635828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.1.244.194:37971)
0807 04:22:18.636460 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.636468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.636504 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:22:18.636615 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.636620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.636702 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.636871 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:18.637310 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.638264 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.640225 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.642016 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.642168 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.642755 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.643547 (+ 792us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.643701 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.643846 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.643954 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":690,"threads_started":1}
I20260807 04:22:18.660386 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.654288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:33469 (local address 127.1.244.194:44821)
0807 04:22:18.654955 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.654987 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.655091 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.655432 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.655435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.655444 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:18.655712 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.655718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.657278 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.657282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.658318 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.658329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.659356 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.659360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.659465 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.659511 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.659951 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.659959 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.660017 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.660115 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":371,"threads_started":1}
I20260807 04:22:18.660403 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.654285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44821 (local address 127.1.244.193:33469)
0807 04:22:18.654493 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.654506 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.655067 (+ 561us) server_negotiation.cc:407] Connection header received
0807 04:22:18.655201 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.655208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.655313 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.655427 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:18.655989 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.657156 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.658458 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.659199 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.659337 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.659652 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.659809 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.659925 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.660058 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.660129 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:22:18.669967 3593 raft_consensus.cc:1261] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Refusing update from remote peer 6a6261c9a0114192b4e91b0e41e8ca61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:18.670269 3316 raft_consensus.cc:1261] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Refusing update from remote peer 6a6261c9a0114192b4e91b0e41e8ca61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:18.670624 3696 consensus_queue.cc:1035] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a478384370f045dbb274e73d904f67af" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:18.670974 3670 consensus_queue.cc:1035] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd65f5a66103494e96016657f11afcfb" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 33469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:18.682215 3679 mvcc.cc:205] Tried to move back new op lower bound from 7315769502386245632 to 7315769501537898496. Current Snapshot: MvccSnapshot[applied={T|T < 7315769502386245632}]
I20260807 04:22:18.711220 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.704633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:44663 (local address 127.1.244.194:37971)
0807 04:22:18.704840 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.704846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.705720 (+ 874us) server_negotiation.cc:407] Connection header received
0807 04:22:18.705910 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.705916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.705996 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.706100 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:18.706674 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.707641 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.709270 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.709836 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.709971 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.710486 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.710608 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.710703 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.710975 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.711036 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 04:22:18.711642 3722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.704688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:37971 (local address 127.1.244.193:44663)
0807 04:22:18.705600 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:18.705623 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:18.705784 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:18.706123 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:18.706131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:18.706148 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:18.706518 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.706529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.707772 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.707777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.709124 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:18.709140 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.710020 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.710027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:18.710229 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.710314 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:18.710724 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:18.710744 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 04:22:18.710888 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 04:22:18.711036 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":338,"threads_started":1}
I20260807 04:22:18.744992 3151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:18.745266 3151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:18.748718 3151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f74a195ba534607a6dc8406a337732e: Sending DeleteTablet for 3 replicas of tablet 3c2fb46ea31d401494440e3d554195c5
I20260807 04:22:18.749657 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2fb46ea31d401494440e3d554195c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:18 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:43842
I20260807 04:22:18.749899 3571 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2fb46ea31d401494440e3d554195c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:18 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:52482
I20260807 04:22:18.750389 3289 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2fb46ea31d401494440e3d554195c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:18 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44248
I20260807 04:22:18.764966 3725 tablet_replica.cc:330] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: stopping tablet replica
I20260807 04:22:18.765205 3725 raft_consensus.cc:2229] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:18.765725 3725 raft_consensus.cc:2258] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:18.766973 3725 ts_tablet_manager.cc:1905] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:18.774693 3725 ts_tablet_manager.cc:1918] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:18.774837 3725 log.cc:1196] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/3c2fb46ea31d401494440e3d554195c5
I20260807 04:22:18.775347 3725 ts_tablet_manager.cc:1939] T 3c2fb46ea31d401494440e3d554195c5 P 6a6261c9a0114192b4e91b0e41e8ca61: Deleting consensus metadata
I20260807 04:22:18.776435 3136 catalog_manager.cc:4841] TS 6a6261c9a0114192b4e91b0e41e8ca61 (127.1.244.194:37971): tablet 3c2fb46ea31d401494440e3d554195c5 (table test-table [id=e860b813d7d748d099204f555d624b87]) successfully deleted
I20260807 04:22:18.785998 3727 tablet_replica.cc:330] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: stopping tablet replica
I20260807 04:22:18.786216 3727 raft_consensus.cc:2229] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:18.786520 3727 raft_consensus.cc:2258] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:18.787123 3726 tablet_replica.cc:330] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: stopping tablet replica
I20260807 04:22:18.787325 3726 raft_consensus.cc:2229] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:18.787593 3726 raft_consensus.cc:2258] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:18.788854 3727 ts_tablet_manager.cc:1905] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:18.788869 3726 ts_tablet_manager.cc:1905] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:18.796976 3726 ts_tablet_manager.cc:1918] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:18.797159 3726 log.cc:1196] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/3c2fb46ea31d401494440e3d554195c5
I20260807 04:22:18.797643 3726 ts_tablet_manager.cc:1939] T 3c2fb46ea31d401494440e3d554195c5 P a478384370f045dbb274e73d904f67af: Deleting consensus metadata
I20260807 04:22:18.798785 3135 catalog_manager.cc:4841] TS a478384370f045dbb274e73d904f67af (127.1.244.195:38817): tablet 3c2fb46ea31d401494440e3d554195c5 (table test-table [id=e860b813d7d748d099204f555d624b87]) successfully deleted
I20260807 04:22:18.798830 3727 ts_tablet_manager.cc:1918] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:18.798981 3727 log.cc:1196] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/3c2fb46ea31d401494440e3d554195c5
I20260807 04:22:18.799471 3727 ts_tablet_manager.cc:1939] T 3c2fb46ea31d401494440e3d554195c5 P fd65f5a66103494e96016657f11afcfb: Deleting consensus metadata
I20260807 04:22:18.800619 3137 catalog_manager.cc:4841] TS fd65f5a66103494e96016657f11afcfb (127.1.244.193:33469): tablet 3c2fb46ea31d401494440e3d554195c5 (table test-table [id=e860b813d7d748d099204f555d624b87]) successfully deleted
Aug 07 04:22:18 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:18.820663 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.790761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.1.244.193:33469)
0807 04:22:18.790963 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.790970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.801558 (+ 10588us) server_negotiation.cc:407] Connection header received
0807 04:22:18.801775 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.801782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.801871 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.801979 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:18.804017 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.805250 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.807121 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.807787 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.807943 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.813060 (+ 5117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:18.813077 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:18.813097 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:18.813168 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:18.817351 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.818184 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.818193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.818197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.818363 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.818767 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.818774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.818779 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.819266 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:18.819596 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.819862 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.820115 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 04:22:18.821448 3356 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44274
I20260807 04:22:18.843555 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.825311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.1.244.254:36979)
0807 04:22:18.825553 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.825560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.828373 (+ 2813us) server_negotiation.cc:407] Connection header received
0807 04:22:18.828524 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.828530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.828611 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.828710 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:18.830349 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.831469 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.833346 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.834061 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.834188 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.837147 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:18.837154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:18.837159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:18.837198 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:18.840570 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.841325 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.841331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.841335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.841473 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.841907 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.841912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.841915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.842268 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:18.842517 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.842920 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.843188 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 04:22:18.844226 3151 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36154
I20260807 04:22:18.846305 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3223
I20260807 04:22:18.867558 3662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.848540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.1.244.193:33469)
0807 04:22:18.848745 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.848751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.852333 (+ 3582us) server_negotiation.cc:407] Connection header received
0807 04:22:18.852500 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.852506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.852597 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.852691 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:18.854120 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.855298 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.857114 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.857848 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.857986 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.860704 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:18.860712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:18.860718 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:18.860756 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:18.864511 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.865288 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.865294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.865298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.865492 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.865883 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.865895 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.865898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.866204 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:18.866634 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.866799 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.867038 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 04:22:18.868144 3356 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44278
W20260807 04:22:18.868793 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3223: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:18.868976 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3223
I20260807 04:22:18.902949 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3365
Aug 07 04:22:18 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:18.919073 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.903726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.1.244.194:37971)
0807 04:22:18.903965 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.903971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.903993 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:22:18.904156 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.904163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.904271 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.904397 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:18.905605 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.906835 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.908048 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.908713 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.908842 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.913051 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:18.913064 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:18.913077 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:18.913108 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:18.916349 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.917133 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.917140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.917142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.917282 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.917659 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.917662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.917664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.917998 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:18.918257 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.918543 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.918701 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 04:22:18.919734 3496 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43858
W20260807 04:22:18.920425 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3365: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:18.920586 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3365
I20260807 04:22:18.941896 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3505
Aug 07 04:22:18 dist-test-slave-zkpd krb5kdc[3106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076537, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:18.957387 3660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:18.942665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52492 (local address 127.1.244.195:38817)
0807 04:22:18.942829 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:18.942833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:18.942850 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:22:18.943022 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:18.943026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:18.943084 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:18.943162 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:18.944661 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.945370 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.947132 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:18.947604 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:18.947691 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:18.952231 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:18.952242 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:18.952256 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:18.952284 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:18.954862 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.955465 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.955471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.955473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.955608 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:18.956141 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:18.956144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:18.956146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:18.956464 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:18.956699 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:18.956888 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 04:22:18.957044 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 04:22:18.958063 3639 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52492
W20260807 04:22:18.958653 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3505: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:18.958777 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3505
I20260807 04:22:18.977460 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3119
I20260807 04:22:18.978468 3192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:19.092129 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3119
2026-08-07T04:22:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4540 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:19 dist-test-slave-zkpd krb5kdc[3738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:19 dist-test-slave-zkpd krb5kdc[3738](info): set up 2 sockets
Aug 07 04:22:19 dist-test-slave-zkpd krb5kdc[3738](info): commencing operation
krb5kdc: starting...
W20260807 04:22:20.202862 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:20 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:20.391157 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42571
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:20.709501 3751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:20.709792 3751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:20.709903 3751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:20.717582 3751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:20.717695 3751 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 04:22:20.717792 3751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:20.717891 3751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:20.717971 3751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:20.718053 3751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:20.727510 3751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42571
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:20.728722 3751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:20.729987 3751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:20.741468 3757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:20.741509 3759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:20.741534 3756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:20.742059 3751 server_base.cc:1034] running on GCE node
I20260807 04:22:20.742844 3751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:20.744103 3751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:20.745318 3751 hybrid_clock.cc:648] HybridClock initialized: now 1786076540745257 us; error 31 us; skew 500 ppm
I20260807 04:22:20.745632 3751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:20 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076540, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:20.748638 3751 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:20.749938 3751 webserver.cc:460] Webserver started at http://127.1.244.254:42945/ using document root <none> and password file <none>
I20260807 04:22:20.750527 3751 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:20.750609 3751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:20.750849 3751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:20.753695 3751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "1b27487f08a043ae9d02ae4bac138482"
format_stamp: "Formatted at 2026-08-07 04:22:20 on dist-test-slave-zkpd"
server_key: "370e79ae2c2429bc8d979f77b944ab1f"
server_key_iv: "ec3ba9797d874629368fab3345db98ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:20.754391 3751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1b27487f08a043ae9d02ae4bac138482"
format_stamp: "Formatted at 2026-08-07 04:22:20 on dist-test-slave-zkpd"
server_key: "370e79ae2c2429bc8d979f77b944ab1f"
server_key_iv: "ec3ba9797d874629368fab3345db98ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:20.759301 3751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:22:20.762486 3766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:20.763695 3751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:22:20.763859 3751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "1b27487f08a043ae9d02ae4bac138482"
format_stamp: "Formatted at 2026-08-07 04:22:20 on dist-test-slave-zkpd"
server_key: "370e79ae2c2429bc8d979f77b944ab1f"
server_key_iv: "ec3ba9797d874629368fab3345db98ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:20.764106 3751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:20.780220 3751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:20.784240 3751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:20.784502 3751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:20.805562 3751 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42571
I20260807 04:22:20.805580 3829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42571 every 8 connection(s)
I20260807 04:22:20.806880 3751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:20.809593 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3751
I20260807 04:22:20.809839 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:20.810664 2003 external_mini_cluster.cc:1428] Setting key 1d245384060e0396a7bdb55d936e8135
I20260807 04:22:20.811239 3830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:20.825649 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Bootstrap starting.
Aug 07 04:22:20 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:20.828835 3830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:20.829885 3830 log.cc:824] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:20.832693 3830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: No bootstrap required, opened a new log
I20260807 04:22:20.834046 3833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:20.812727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.1.244.254:42571)
0807 04:22:20.813464 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:20.813475 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:20.813513 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:22:20.815784 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:20.815805 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:20.816241 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:20.816581 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:20.818345 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:20.819608 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:20.821394 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:20.822080 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:20.822214 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:20.826981 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:20.826991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:20.827005 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:20.827043 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:20.830184 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:20.830885 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:20.830896 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:20.830905 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:20.831062 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:20.831425 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:20.831434 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:20.831436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:20.831999 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:20.832279 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:20.832722 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 04:22:20.833107 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":352,"threads_started":1}
I20260807 04:22:20.840886 3830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b27487f08a043ae9d02ae4bac138482" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42571 } }
I20260807 04:22:20.841243 3830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:20.841343 3830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b27487f08a043ae9d02ae4bac138482, State: Initialized, Role: FOLLOWER
I20260807 04:22:20.841845 3830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b27487f08a043ae9d02ae4bac138482" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42571 } }
I20260807 04:22:20.842080 3830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:20.842213 3830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:20.842378 3830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:20.844682 3830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b27487f08a043ae9d02ae4bac138482" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42571 } }
I20260807 04:22:20.845158 3830 leader_election.cc:302] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b27487f08a043ae9d02ae4bac138482; no voters:
I20260807 04:22:20.845747 3830 leader_election.cc:288] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:20.845871 3835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:20.846555 3835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [term 1 LEADER]: Becoming Leader. State: Replica: 1b27487f08a043ae9d02ae4bac138482, State: Running, Role: LEADER
I20260807 04:22:20.847080 3835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b27487f08a043ae9d02ae4bac138482" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42571 } }
I20260807 04:22:20.847743 3830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:20.849804 3837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b27487f08a043ae9d02ae4bac138482. Latest consensus state: current_term: 1 leader_uuid: "1b27487f08a043ae9d02ae4bac138482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b27487f08a043ae9d02ae4bac138482" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42571 } } }
I20260807 04:22:20.849781 3836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b27487f08a043ae9d02ae4bac138482" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b27487f08a043ae9d02ae4bac138482" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42571 } } }
I20260807 04:22:20.850208 3837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:20.850208 3836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:20.851944 3843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:20.857242 3843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:20.866181 3843 catalog_manager.cc:1347] Generated new cluster ID: 5ad93fe24bac452d9ea35e1f6275ec28
I20260807 04:22:20.866281 3843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:20.882943 3843 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:20.884241 3843 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:20.889456 3843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Generated new TSK 0
I20260807 04:22:20.890341 3843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:20.943739 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42571
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:21.268366 3857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:21.268671 3857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:21.268827 3857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:21.275945 3857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:21.276089 3857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:21.276233 3857 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 04:22:21.276319 3857 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:22:21.276440 3857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:21.285890 3857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:42571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:21.287127 3857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:21.288568 3857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:21.307283 3862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:21.307531 3863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:21.307968 3865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:21.308806 3857 server_base.cc:1034] running on GCE node
I20260807 04:22:21.309365 3857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:21.310324 3857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:21.311545 3857 hybrid_clock.cc:648] HybridClock initialized: now 1786076541311499 us; error 39 us; skew 500 ppm
I20260807 04:22:21.311851 3857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:21 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:21.314942 3857 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:21.316329 3857 webserver.cc:460] Webserver started at http://127.1.244.193:34543/ using document root <none> and password file <none>
I20260807 04:22:21.316998 3857 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:21.317139 3857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:21.317458 3857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:21.320297 3857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4"
format_stamp: "Formatted at 2026-08-07 04:22:21 on dist-test-slave-zkpd"
server_key: "5d611a4be36bae2b5278625a53184792"
server_key_iv: "c7597890b076f0e3f8814840e2cd546d"
server_key_version: "encryptionkey@0"
I20260807 04:22:21.321087 3857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4"
format_stamp: "Formatted at 2026-08-07 04:22:21 on dist-test-slave-zkpd"
server_key: "5d611a4be36bae2b5278625a53184792"
server_key_iv: "c7597890b076f0e3f8814840e2cd546d"
server_key_version: "encryptionkey@0"
I20260807 04:22:21.325965 3857 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 04:22:21.329470 3872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:21.330719 3857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:22:21.330922 3857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4"
format_stamp: "Formatted at 2026-08-07 04:22:21 on dist-test-slave-zkpd"
server_key: "5d611a4be36bae2b5278625a53184792"
server_key_iv: "c7597890b076f0e3f8814840e2cd546d"
server_key_version: "encryptionkey@0"
I20260807 04:22:21.331174 3857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:21.350273 3857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:21.354684 3857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:21.354977 3857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:21.357686 3857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:21.357843 3857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:21.357970 3857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:21.358050 3857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:22:21 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:21.379977 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.358084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35173 (local address 127.1.244.254:42571)
0807 04:22:21.358973 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:21.358978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:21.359646 (+ 668us) server_negotiation.cc:407] Connection header received
0807 04:22:21.361667 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:21.361673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:21.361762 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:21.361876 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:21.364058 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.365125 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.367994 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.368685 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.368862 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.373353 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:21.373358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:21.373362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:21.373393 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:21.376783 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.377697 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.377703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.377706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.377834 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.378298 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.378303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.378306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.378571 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:21.378719 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:21.379340 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 04:22:21.379553 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":684,"threads_started":1}
I20260807 04:22:21.380993 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.358699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.1.244.193:35173)
0807 04:22:21.359509 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:21.359544 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:21.361414 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:21.362098 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:21.362106 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:21.362870 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:21.363822 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.363853 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.365295 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.365307 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.367772 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.367791 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.368978 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.368987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.369255 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.369974 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:21.369998 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:21.373056 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:21.376954 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.376960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.376971 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.377566 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.377979 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.377985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.377988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.378183 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.378695 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:21.378702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:21.379012 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 04:22:21.379242 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 04:22:21.379717 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":321,"threads_started":1}
I20260807 04:22:21.404341 3944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.387304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.1.244.193:35871)
0807 04:22:21.388131 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:21.388157 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:21.388331 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:21.388629 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:21.388634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:21.389404 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:21.389937 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.389956 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.391685 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.391690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.393001 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.393015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.394215 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.394225 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.394456 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.395157 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:21.395199 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:21.396490 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:21.400602 (+ 4112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.400609 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.400615 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.401173 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.401683 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.401689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.401695 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.401885 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.402631 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:21.402638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:21.402919 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 04:22:21.403104 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 04:22:21.403589 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":467,"threads_started":1}
I20260807 04:22:21.405416 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.387265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35871 (local address 127.1.244.254:42571)
0807 04:22:21.387437 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:21.387443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:21.388263 (+ 820us) server_negotiation.cc:407] Connection header received
0807 04:22:21.388441 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:21.388444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:21.388498 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:21.388604 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:21.390118 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.391494 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.393152 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.393989 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.394138 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.396696 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:21.396704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:21.396712 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:21.396753 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:21.400428 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.401313 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.401321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.401327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.401525 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.402049 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.402056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.402060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.402479 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:21.402653 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:21.404740 (+ 2087us) server_negotiation.cc:299] Negotiation successful
0807 04:22:21.405010 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:22:21.408526 3857 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:37553
I20260807 04:22:21.408612 3994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:37553 every 8 connection(s)
I20260807 04:22:21.410104 3857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:21.412098 3995 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42571
I20260807 04:22:21.412315 3995 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:21.412515 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3857
I20260807 04:22:21.412717 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:21.413008 3995 heartbeater.cc:511] Master 127.1.244.254:42571 requested a full tablet report, sending...
I20260807 04:22:21.413384 2003 external_mini_cluster.cc:1428] Setting key 774b3061c94184017852487079326db8
I20260807 04:22:21.415184 3784 ts_manager.cc:194] Registered new tserver with Master: 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553)
I20260807 04:22:21.416703 3784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:35173
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:21.455061 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42571
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:21.775619 3999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:21.775939 3999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:21.776106 3999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:21.783033 3999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:21.783161 3999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:21.783293 3999 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 04:22:21.783392 3999 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:22:21.783464 3999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:21.792652 3999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:42571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:21.793768 3999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:21.794998 3999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:21.807629 4004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:21.807663 4005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:21.808245 4007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:21.809010 3999 server_base.cc:1034] running on GCE node
I20260807 04:22:21.809630 3999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:21.810636 3999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:21.811952 3999 hybrid_clock.cc:648] HybridClock initialized: now 1786076541811866 us; error 48 us; skew 500 ppm
I20260807 04:22:21.812361 3999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:21 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:21.815781 3999 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:21.817471 3999 webserver.cc:460] Webserver started at http://127.1.244.194:37927/ using document root <none> and password file <none>
I20260807 04:22:21.818117 3999 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:21.818257 3999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:21.818576 3999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:21.821307 3999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "b5f30e6489f6443b9e271db87cb5fd14"
format_stamp: "Formatted at 2026-08-07 04:22:21 on dist-test-slave-zkpd"
server_key: "208d3d703fd818dd918cacda2c484ec7"
server_key_iv: "440d4c28c51fc3e3ffb3c30c5cf00713"
server_key_version: "encryptionkey@0"
I20260807 04:22:21.822190 3999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "b5f30e6489f6443b9e271db87cb5fd14"
format_stamp: "Formatted at 2026-08-07 04:22:21 on dist-test-slave-zkpd"
server_key: "208d3d703fd818dd918cacda2c484ec7"
server_key_iv: "440d4c28c51fc3e3ffb3c30c5cf00713"
server_key_version: "encryptionkey@0"
I20260807 04:22:21.828629 3999 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 04:22:21.832062 4014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:21.833278 3999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:22:21.833444 3999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "b5f30e6489f6443b9e271db87cb5fd14"
format_stamp: "Formatted at 2026-08-07 04:22:21 on dist-test-slave-zkpd"
server_key: "208d3d703fd818dd918cacda2c484ec7"
server_key_iv: "440d4c28c51fc3e3ffb3c30c5cf00713"
server_key_version: "encryptionkey@0"
I20260807 04:22:21.833626 3999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:21.865756 3999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:21.869992 3999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:21.870275 3999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:21.872941 3999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:21.873032 3999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:21.873147 3999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:21.873234 3999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:22:21 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076541, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:21.899951 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.873252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:54003 (local address 127.1.244.254:42571)
0807 04:22:21.873467 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:21.873476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:21.874952 (+ 1476us) server_negotiation.cc:407] Connection header received
0807 04:22:21.878016 (+ 3064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:21.878020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:21.878135 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:21.878266 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:21.880951 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.882059 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.885486 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.886038 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.886198 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.892540 (+ 6342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:21.892547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:21.892552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:21.892593 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:21.896140 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.897344 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.897348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.897351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.897487 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.898089 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.898092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.898094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.898351 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:21.898486 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:21.899496 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0807 04:22:21.899672 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:22:21.902513 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.873858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.1.244.194:54003)
0807 04:22:21.874749 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:21.874820 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:21.877672 (+ 2852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:21.878559 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:21.878576 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:21.879536 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:21.880636 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.880672 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.882276 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.882285 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.885234 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.885258 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.886378 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.886389 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.886771 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.887570 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:21.887610 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:21.892250 (+ 4640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:21.896378 (+ 4128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.896391 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.896413 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.897163 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.897598 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.897605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.897611 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.897967 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.898498 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:21.898509 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:21.899045 (+ 536us) client_negotiation.cc:769] Sending connection context
0807 04:22:21.899413 (+ 368us) client_negotiation.cc:240] Negotiation successful
0807 04:22:21.900298 (+ 885us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":334,"threads_started":1}
I20260807 04:22:21.922027 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.908157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:57835 (local address 127.1.244.254:42571)
0807 04:22:21.908325 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:21.908330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:21.909414 (+ 1084us) server_negotiation.cc:407] Connection header received
0807 04:22:21.909639 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:21.909646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:21.909726 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:21.909855 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:21.910988 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.911710 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.913492 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.914281 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.914435 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.916070 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:21.916077 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:21.916082 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:21.916120 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:21.919100 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.920137 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.920140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.920142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.920263 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:21.920693 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:21.920696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:21.920698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:21.921006 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:21.921111 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:21.921658 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 04:22:21.921804 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:22:21.922665 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:21.908318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.1.244.194:57835)
0807 04:22:21.909272 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:21.909298 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:21.909480 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:21.909880 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:21.909885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:21.910583 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:21.910853 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.910862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.911834 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.911839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.913330 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:21.913351 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:21.914471 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:21.914476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:21.914581 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:21.915168 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:21.915185 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:21.915860 (+ 675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:21.919248 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.919256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.919263 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.920002 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.920375 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:21.920383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:21.920389 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:21.920579 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:21.921116 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:21.921124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:21.921381 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 04:22:21.921571 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 04:22:21.922000 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":495,"threads_started":1}
I20260807 04:22:21.931437 3999 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42099
I20260807 04:22:21.931514 4135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42099 every 8 connection(s)
I20260807 04:22:21.932708 3999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:21.934839 4136 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42571
I20260807 04:22:21.934834 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 3999
I20260807 04:22:21.935032 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:21.935081 4136 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:21.935491 2003 external_mini_cluster.cc:1428] Setting key 0aa7175a15f232f7bba686f0066264ed
I20260807 04:22:21.935765 4136 heartbeater.cc:511] Master 127.1.244.254:42571 requested a full tablet report, sending...
I20260807 04:22:21.937363 3783 ts_manager.cc:194] Registered new tserver with Master: b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099)
I20260807 04:22:21.938513 3783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:54003
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:21.979125 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42571
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:22.307301 4140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:22.307546 4140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:22.307657 4140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:22.314638 4140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:22.314743 4140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:22.314826 4140 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 04:22:22.314879 4140 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:22:22.314946 4140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:22.324440 4140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:42571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:22.325639 4140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:22.327023 4140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:22.340029 4145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:22.339994 4146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:22.340188 4148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:22.340848 4140 server_base.cc:1034] running on GCE node
I20260807 04:22:22.341432 4140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:22.342234 4140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:22.343425 4140 hybrid_clock.cc:648] HybridClock initialized: now 1786076542343369 us; error 50 us; skew 500 ppm
I20260807 04:22:22.343708 4140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:22 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:22.350839 4140 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:22.352317 4140 webserver.cc:460] Webserver started at http://127.1.244.195:36251/ using document root <none> and password file <none>
I20260807 04:22:22.353017 4140 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:22.353159 4140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:22.353470 4140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:22.356359 4140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "62e99f129ff34c249544f483d0feb183"
format_stamp: "Formatted at 2026-08-07 04:22:22 on dist-test-slave-zkpd"
server_key: "b089ecb4868d72e21be8c9b7149ec850"
server_key_iv: "762db719989ac7587ab429c54b523529"
server_key_version: "encryptionkey@0"
I20260807 04:22:22.357105 4140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "62e99f129ff34c249544f483d0feb183"
format_stamp: "Formatted at 2026-08-07 04:22:22 on dist-test-slave-zkpd"
server_key: "b089ecb4868d72e21be8c9b7149ec850"
server_key_iv: "762db719989ac7587ab429c54b523529"
server_key_version: "encryptionkey@0"
I20260807 04:22:22.362038 4140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 04:22:22.365283 4155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:22.366454 4140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 04:22:22.366622 4140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "62e99f129ff34c249544f483d0feb183"
format_stamp: "Formatted at 2026-08-07 04:22:22 on dist-test-slave-zkpd"
server_key: "b089ecb4868d72e21be8c9b7149ec850"
server_key_iv: "762db719989ac7587ab429c54b523529"
server_key_version: "encryptionkey@0"
I20260807 04:22:22.366843 4140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:22.386598 4140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:22.390657 4140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:22.390957 4140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:22.393867 4140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:22.394008 4140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:22.394142 4140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:22.394243 4140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:22:22 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076542, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:22.418701 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.394378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54039 (local address 127.1.244.254:42571)
0807 04:22:22.394580 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.394587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.395982 (+ 1395us) server_negotiation.cc:407] Connection header received
0807 04:22:22.398042 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.398048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.398141 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.398250 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:22.400577 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.401764 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.405108 (+ 3344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.405733 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.405866 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.410016 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:22.410022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:22.410028 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:22.410068 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:22.413785 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.414785 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.414793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.414799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.415001 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.415714 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.415722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.415727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.416195 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:22.416397 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.417839 (+ 1442us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.418161 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 04:22:22.419009 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.395018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.1.244.195:54039)
0807 04:22:22.395802 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.395839 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.397765 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.398488 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.398496 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.399311 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:22.400295 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.400327 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.401970 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.401981 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.404924 (+ 2943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.404946 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.405997 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.406007 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.406191 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.406847 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:22.406872 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:22.409731 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:22.413960 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:22.413969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:22.413983 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:22.414626 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:22.415171 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:22.415180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:22.415187 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:22.415520 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:22.416341 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:22.416354 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:22.416773 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.417130 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.417629 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":361,"threads_started":1}
I20260807 04:22:22.419488 3995 heartbeater.cc:503] Master 127.1.244.254:42571 was elected leader, sending a full tablet report...
I20260807 04:22:22.441929 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.426693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:33357 (local address 127.1.244.254:42571)
0807 04:22:22.426881 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.426897 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.427205 (+ 308us) server_negotiation.cc:407] Connection header received
0807 04:22:22.427446 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.427452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.427545 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.427651 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:22.429091 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.430002 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.432053 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.432588 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.432709 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.435138 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:22.435145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:22.435153 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:22.435189 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:22.438810 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.439703 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.439710 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.439714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.439863 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.440321 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.440327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.440330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.440711 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:22.440897 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.441323 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.441550 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:22:22.442057 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.426389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.1.244.195:33357)
0807 04:22:22.427096 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.427124 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.427300 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.427692 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.427697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.428387 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:22.428876 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.428905 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.430213 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.430223 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.431821 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.431832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.433018 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.433023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.433136 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.433895 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:22.433915 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:22.434844 (+ 929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:22.438932 (+ 4088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:22.438937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:22.438941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:22.439569 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:22.440051 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:22.440056 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:22.440059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:22.440214 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:22.440822 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:22.440827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:22.441024 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.441187 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.441550 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":511,"threads_started":1}
I20260807 04:22:22.444964 4140 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:44607
I20260807 04:22:22.445096 4275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:44607 every 8 connection(s)
I20260807 04:22:22.446458 4140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:22.448431 4276 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42571
I20260807 04:22:22.448666 4276 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:22.449357 4276 heartbeater.cc:511] Master 127.1.244.254:42571 requested a full tablet report, sending...
I20260807 04:22:22.449801 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 4140
I20260807 04:22:22.449992 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:22.450493 2003 external_mini_cluster.cc:1428] Setting key 9aa3c69eaca758c831c2e39d3eb4e27a
I20260807 04:22:22.450803 3784 ts_manager.cc:194] Registered new tserver with Master: 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:22.451824 3784 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:54039
I20260807 04:22:22.454442 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:22:22 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:22:22 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:22.501261 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.488629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.1.244.254:42571)
0807 04:22:22.488832 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.488839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.489028 (+ 189us) server_negotiation.cc:407] Connection header received
0807 04:22:22.489192 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.489196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.489258 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.489337 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:22.490637 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.491356 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.492535 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.492920 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.493007 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.496622 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:22.496627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:22.496631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:22.496662 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:22.499084 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.499631 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.499634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.499637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.499751 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.500108 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.500111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.500113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.500345 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:22.500520 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.500752 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.500905 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:22:22.505106 3784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:22.511765 3784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:22.534129 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.524192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.1.244.194:42099)
0807 04:22:22.525336 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.525345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.526918 (+ 1573us) server_negotiation.cc:407] Connection header received
0807 04:22:22.527045 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.527056 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.527353 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.527565 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.528568 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.529771 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.532052 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.532913 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.533053 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.533269 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.533621 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.533823 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":829,"threads_started":1}
I20260807 04:22:22.536264 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.523595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.1.244.193:37553)
0807 04:22:22.524908 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.524916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.526160 (+ 1244us) server_negotiation.cc:407] Connection header received
0807 04:22:22.527370 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.527378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.527624 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.527802 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.528920 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.530407 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.533199 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.534961 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.535102 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.535331 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.535728 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.535940 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1155,"thread_start_us":1010,"threads_started":1}
I20260807 04:22:22.536296 4290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.523319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:37553 (local address 127.0.0.1:33154)
0807 04:22:22.525972 (+ 2653us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.526032 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.526348 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.528103 (+ 1755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.528108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.528141 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.528701 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.528717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.530592 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.530597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.533013 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.533028 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.535365 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.535375 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.535539 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.535567 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.535644 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.535844 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2309,"mutex_wait_us":50,"thread_start_us":3344,"threads_started":1}
I20260807 04:22:22.537153 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.525803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47374 (local address 127.1.244.195:44607)
0807 04:22:22.528124 (+ 2321us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.528133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.528164 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:22:22.528262 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.528271 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.528486 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.528690 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.529286 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.530872 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.533458 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.535463 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.535580 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.535836 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.536645 (+ 809us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.536825 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2153,"thread_start_us":1954,"threads_started":1}
I20260807 04:22:22.537170 4289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.525573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44607 (local address 127.0.0.1:47374)
0807 04:22:22.526957 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.526976 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.527103 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.528674 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.528684 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.528708 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.529073 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.529086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.531035 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.531040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.533277 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.533293 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.536030 (+ 2737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.536037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.536222 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.536264 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.536358 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.536794 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"mutex_wait_us":2,"thread_start_us":674,"threads_started":1}
I20260807 04:22:22.537545 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.523577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42099 (local address 127.0.0.1:37402)
0807 04:22:22.526586 (+ 3009us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.526607 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.526744 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.528014 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.528019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.528042 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.528394 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.528403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.529904 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.529910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.531865 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.531880 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.533114 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.533120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.533326 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.533414 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.536537 (+ 3123us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.536861 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2853,"mutex_wait_us":186,"thread_start_us":1905,"threads_started":1}
I20260807 04:22:22.540812 4205 tablet_service.cc:1467] Processing CreateTablet for tablet ea936869683448c98dcf75973ae0303b (DEFAULT_TABLE table=test-table [id=040d6e2b68f44d51b29a29e6e3b0a96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:22.540827 3924 tablet_service.cc:1467] Processing CreateTablet for tablet ea936869683448c98dcf75973ae0303b (DEFAULT_TABLE table=test-table [id=040d6e2b68f44d51b29a29e6e3b0a96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:22.540870 4065 tablet_service.cc:1467] Processing CreateTablet for tablet ea936869683448c98dcf75973ae0303b (DEFAULT_TABLE table=test-table [id=040d6e2b68f44d51b29a29e6e3b0a96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:22.542120 4205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea936869683448c98dcf75973ae0303b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:22.542192 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea936869683448c98dcf75973ae0303b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:22.542197 4065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea936869683448c98dcf75973ae0303b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:22.554774 4295 tablet_bootstrap.cc:492] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Bootstrap starting.
I20260807 04:22:22.556630 4296 tablet_bootstrap.cc:492] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Bootstrap starting.
I20260807 04:22:22.558761 4295 tablet_bootstrap.cc:654] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:22.560138 4296 tablet_bootstrap.cc:654] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:22.560094 4295 log.cc:824] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:22.560660 4297 tablet_bootstrap.cc:492] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Bootstrap starting.
I20260807 04:22:22.561622 4296 log.cc:824] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:22.563591 4297 tablet_bootstrap.cc:654] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:22.564810 4297 log.cc:824] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:22.566606 4296 tablet_bootstrap.cc:492] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: No bootstrap required, opened a new log
I20260807 04:22:22.566620 4295 tablet_bootstrap.cc:492] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: No bootstrap required, opened a new log
I20260807 04:22:22.566890 4296 ts_tablet_manager.cc:1397] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 04:22:22.566907 4295 ts_tablet_manager.cc:1397] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 04:22:22.567200 4297 tablet_bootstrap.cc:492] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: No bootstrap required, opened a new log
I20260807 04:22:22.567467 4297 ts_tablet_manager.cc:1397] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 04:22:22.574918 4297 raft_consensus.cc:348] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.575350 4297 raft_consensus.cc:374] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:22.575469 4297 raft_consensus.cc:729] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f30e6489f6443b9e271db87cb5fd14, State: Initialized, Role: FOLLOWER
I20260807 04:22:22.576081 4297 consensus_queue.cc:260] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.577059 4136 heartbeater.cc:503] Master 127.1.244.254:42571 was elected leader, sending a full tablet report...
I20260807 04:22:22.577678 4296 raft_consensus.cc:348] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.577678 4295 raft_consensus.cc:348] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.578086 4295 raft_consensus.cc:374] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:22.578089 4296 raft_consensus.cc:374] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:22.578181 4296 raft_consensus.cc:729] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Initialized, Role: FOLLOWER
I20260807 04:22:22.578186 4295 raft_consensus.cc:729] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Initialized, Role: FOLLOWER
I20260807 04:22:22.578792 4297 ts_tablet_manager.cc:1428] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260807 04:22:22.578734 4295 consensus_queue.cc:260] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.578822 4296 consensus_queue.cc:260] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.580510 4276 heartbeater.cc:503] Master 127.1.244.254:42571 was elected leader, sending a full tablet report...
I20260807 04:22:22.580737 4295 ts_tablet_manager.cc:1428] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260807 04:22:22.581064 4296 ts_tablet_manager.cc:1428] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260807 04:22:22.651374 4303 raft_consensus.cc:482] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:22.651707 4303 raft_consensus.cc:504] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.653429 4303 leader_election.cc:288] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099), 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:22.659623 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.653877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:37237 (local address 127.1.244.194:42099)
0807 04:22:22.654129 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.654134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.654471 (+ 337us) server_negotiation.cc:407] Connection header received
0807 04:22:22.654696 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.654700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.654759 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.654849 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.655435 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.656216 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.657946 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.658685 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.658867 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.658942 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.659219 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.659347 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 04:22:22.659777 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.653675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42099 (local address 127.1.244.193:37237)
0807 04:22:22.654358 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.654374 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.654544 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.654870 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.654875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.654909 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.655286 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.655296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.656333 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.656341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.657787 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.657803 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.658900 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.658906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.659060 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.659084 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.659193 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.659323 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"spinlock_wait_cycles":22784,"thread_start_us":324,"threads_started":1}
I20260807 04:22:22.659972 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.653888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:39007 (local address 127.1.244.195:44607)
0807 04:22:22.654140 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.654148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.654953 (+ 805us) server_negotiation.cc:407] Connection header received
0807 04:22:22.655151 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.655156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.655214 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.655309 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.655832 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.656565 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.658252 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.659004 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.659185 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.659261 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.659534 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.659657 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 04:22:22.660115 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.653675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44607 (local address 127.1.244.193:39007)
0807 04:22:22.654824 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.654839 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.654994 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.655326 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.655330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.655352 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.655671 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.655679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.656708 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.656721 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.658102 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.658114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.659223 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.659229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.659373 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.659396 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.659510 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.659637 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":461,"thread_start_us":358,"threads_started":1}
I20260807 04:22:22.660960 4085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea936869683448c98dcf75973ae0303b" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14" is_pre_election: true
I20260807 04:22:22.660974 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea936869683448c98dcf75973ae0303b" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e99f129ff34c249544f483d0feb183" is_pre_election: true
I20260807 04:22:22.661435 4085 raft_consensus.cc:2454] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 0.
I20260807 04:22:22.661439 4229 raft_consensus.cc:2454] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 0.
I20260807 04:22:22.662053 3874 leader_election.cc:302] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4, b5f30e6489f6443b9e271db87cb5fd14; no voters:
I20260807 04:22:22.662459 4303 raft_consensus.cc:2781] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 04:22:22.662465 3996 tablet.cc:2369] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:22:22.662626 4303 raft_consensus.cc:482] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:22.662748 4303 raft_consensus.cc:3037] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:22.664984 4303 raft_consensus.cc:504] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.665596 4303 leader_election.cc:288] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 election: Requested vote from peers b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099), 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:22.665925 4085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea936869683448c98dcf75973ae0303b" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14"
I20260807 04:22:22.666040 4229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea936869683448c98dcf75973ae0303b" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e99f129ff34c249544f483d0feb183"
I20260807 04:22:22.666131 4085 raft_consensus.cc:3037] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:22.666225 4229 raft_consensus.cc:3037] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:22.669409 4085 raft_consensus.cc:2454] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 1.
I20260807 04:22:22.669417 4229 raft_consensus.cc:2454] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 1.
I20260807 04:22:22.669835 3874 leader_election.cc:302] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4; no voters:
I20260807 04:22:22.670120 4303 raft_consensus.cc:2781] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:22.670586 4303 raft_consensus.cc:686] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Running, Role: LEADER
I20260807 04:22:22.671093 4303 consensus_queue.cc:237] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.675532 3783 catalog_manager.cc:5495] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } health_report { overall_health: UNKNOWN } } }
W20260807 04:22:22.684631 4137 tablet.cc:2369] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:22:22.698208 4277 tablet.cc:2369] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 04:22:22 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076540, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:22.747421 3880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.734244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.1.244.254:42571)
0807 04:22:22.734400 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.734414 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.734938 (+ 524us) server_negotiation.cc:407] Connection header received
0807 04:22:22.735139 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.735143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.735228 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.735331 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:22.736597 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.737432 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.738726 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.739156 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.739289 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.742555 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:22.742560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:22.742563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:22.742590 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:22.744816 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.745523 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.745530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.745533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.745661 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:22.746005 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:22.746012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:22.746015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:22.746275 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:22.746419 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.746810 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.747124 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":53}
I20260807 04:22:22.747464 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.734290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42571 (local address 127.0.0.1:56692)
0807 04:22:22.734833 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.734848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.735015 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.735379 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.735383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.736144 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:22.736445 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.736454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.737550 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.737558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.738591 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.738602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.739327 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.739333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.739498 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.740288 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:22.740306 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:22.742380 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:22.744994 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:22.745006 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:22.745014 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:22.745414 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:22.745761 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:22.745767 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:22.745771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:22.745907 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:22.746401 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:22.746409 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:22.746568 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.746722 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.746990 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":303,"threads_started":1}
I20260807 04:22:22.751973 3783 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:56692:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 04:22:22.753005 3783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:22.759066 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 80a0875826064699948060b463caa50b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bb26544448e44d0943de6d6426221d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:22:22.759277 4065 tablet_service.cc:1467] Processing CreateTablet for tablet 80a0875826064699948060b463caa50b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bb26544448e44d0943de6d6426221d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:22:22.760061 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a0875826064699948060b463caa50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:22.760241 4065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a0875826064699948060b463caa50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:22.759783 4205 tablet_service.cc:1467] Processing CreateTablet for tablet 80a0875826064699948060b463caa50b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bb26544448e44d0943de6d6426221d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:22:22.760637 4205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a0875826064699948060b463caa50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:22.765050 4296 tablet_bootstrap.cc:492] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Bootstrap starting.
I20260807 04:22:22.766573 4296 tablet_bootstrap.cc:654] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:22.767696 4297 tablet_bootstrap.cc:492] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: Bootstrap starting.
I20260807 04:22:22.767915 4295 tablet_bootstrap.cc:492] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: Bootstrap starting.
I20260807 04:22:22.769182 4296 tablet_bootstrap.cc:492] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: No bootstrap required, opened a new log
I20260807 04:22:22.769330 4296 ts_tablet_manager.cc:1397] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 04:22:22.769498 4297 tablet_bootstrap.cc:654] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:22.769622 4295 tablet_bootstrap.cc:654] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:22.770183 4296 raft_consensus.cc:348] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.770424 4296 raft_consensus.cc:374] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:22.770555 4296 raft_consensus.cc:729] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Initialized, Role: FOLLOWER
I20260807 04:22:22.770828 4296 consensus_queue.cc:260] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.771402 4303 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.771667 4296 ts_tablet_manager.cc:1428] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:22:22.771728 4303 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.772938 4297 tablet_bootstrap.cc:492] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: No bootstrap required, opened a new log
I20260807 04:22:22.773084 4297 ts_tablet_manager.cc:1397] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 04:22:22.773277 4295 tablet_bootstrap.cc:492] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: No bootstrap required, opened a new log
I20260807 04:22:22.773442 4295 ts_tablet_manager.cc:1397] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 04:22:22.773831 4297 raft_consensus.cc:348] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.774057 4297 raft_consensus.cc:374] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:22.774153 4297 raft_consensus.cc:729] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f30e6489f6443b9e271db87cb5fd14, State: Initialized, Role: FOLLOWER
I20260807 04:22:22.774400 4297 consensus_queue.cc:260] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.774534 4295 raft_consensus.cc:348] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.774776 4295 raft_consensus.cc:374] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:22.774876 4295 raft_consensus.cc:729] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Initialized, Role: FOLLOWER
I20260807 04:22:22.775058 4297 ts_tablet_manager.cc:1428] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:22:22.774892 4301 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.775274 4301 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.775151 4295 consensus_queue.cc:260] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.775631 4302 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.776024 4302 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.776362 4295 ts_tablet_manager.cc:1428] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 04:22:22.886754 4302 raft_consensus.cc:482] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:22.886967 4302 raft_consensus.cc:504] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.888540 4302 leader_election.cc:288] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553), b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099)
I20260807 04:22:22.894165 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.888961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52427 (local address 127.1.244.193:37553)
0807 04:22:22.889181 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.889189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.889285 (+ 96us) server_negotiation.cc:407] Connection header received
0807 04:22:22.889533 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.889537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.889596 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.889699 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.890174 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.890905 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.892537 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.893290 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.893463 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.893537 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.893803 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.893926 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:22:22.894203 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.888917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:37553 (local address 127.1.244.195:52427)
0807 04:22:22.889193 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.889214 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.889346 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.889716 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.889720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.889758 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.890019 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.890027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.891039 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.891045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.892397 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.892408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.893438 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.893446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.893635 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.893658 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.893767 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.893905 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"spinlock_wait_cycles":24320}
I20260807 04:22:22.894917 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.888979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42099 (local address 127.1.244.195:56251)
0807 04:22:22.889625 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.889639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.889816 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.890147 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.890152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.890175 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:22.890526 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.890534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.891581 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.891586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.892928 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.892940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.894106 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.894112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.894337 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.894384 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.894515 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.894630 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":361,"threads_started":1}
I20260807 04:22:22.894965 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.888951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56251 (local address 127.1.244.194:42099)
0807 04:22:22.889139 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.889145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.889766 (+ 621us) server_negotiation.cc:407] Connection header received
0807 04:22:22.889974 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.889979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.890037 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.890132 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:22.890692 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.891440 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.893092 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.893876 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.894063 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.894141 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.894579 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.894723 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:22:22.895236 3950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a0875826064699948060b463caa50b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" is_pre_election: true
I20260807 04:22:22.895593 4085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a0875826064699948060b463caa50b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14" is_pre_election: true
I20260807 04:22:22.895658 3950 raft_consensus.cc:2454] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 0.
I20260807 04:22:22.895846 4085 raft_consensus.cc:2454] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 0.
I20260807 04:22:22.896327 4159 leader_election.cc:302] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4; no voters:
I20260807 04:22:22.896696 4302 raft_consensus.cc:2781] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:22.896829 4302 raft_consensus.cc:482] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:22.896929 4302 raft_consensus.cc:3037] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:22.899163 4302 raft_consensus.cc:504] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.899712 4302 leader_election.cc:288] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 election: Requested vote from peers 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553), b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099)
I20260807 04:22:22.900102 3950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a0875826064699948060b463caa50b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4"
I20260807 04:22:22.900161 4085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a0875826064699948060b463caa50b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14"
I20260807 04:22:22.900283 3950 raft_consensus.cc:3037] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:22.900399 4085 raft_consensus.cc:3037] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:22.902289 4085 raft_consensus.cc:2454] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 1.
I20260807 04:22:22.902349 3950 raft_consensus.cc:2454] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 1.
I20260807 04:22:22.902766 4156 leader_election.cc:302] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, b5f30e6489f6443b9e271db87cb5fd14; no voters:
I20260807 04:22:22.903041 4302 raft_consensus.cc:2781] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:22.903405 4302 raft_consensus.cc:686] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [term 1 LEADER]: Becoming Leader. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Running, Role: LEADER
I20260807 04:22:22.903967 4302 consensus_queue.cc:237] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:22.907344 4318 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: TxnStatusTablet state changed. Reason: New leader 62e99f129ff34c249544f483d0feb183. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.907742 4318 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:22:22.908473 4320 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 04:22:22.908752 4320 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 04:22:22.908757 3783 catalog_manager.cc:5495] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62e99f129ff34c249544f483d0feb183 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } health_report { overall_health: HEALTHY } } }
I20260807 04:22:22.929536 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.923503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44607 (local address 127.0.0.1:47382)
0807 04:22:22.923768 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.923789 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.923946 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.924432 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.924435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.924444 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:22.924599 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.924605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.926036 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.926039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.926993 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.927005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.927990 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.927994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.928142 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.928194 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:22.929128 (+ 934us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:22.929143 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.929198 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.929314 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:22:22.929636 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.923705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.1.244.195:44607)
0807 04:22:22.923940 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.923946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.923969 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:22:22.924144 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.924150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.924262 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.924434 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:22.924724 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.925924 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.927141 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.927804 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.927969 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.928369 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:22.929011 (+ 642us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:22.929152 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.929227 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.929312 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:22:22.938961 3950 raft_consensus.cc:1261] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Refusing update from remote peer 62e99f129ff34c249544f483d0feb183: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:22.938989 4085 raft_consensus.cc:1261] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Refusing update from remote peer 62e99f129ff34c249544f483d0feb183: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:22.940055 4318 consensus_queue.cc:1035] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:22.940716 4302 consensus_queue.cc:1035] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:22.956329 4303 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: TxnStatusTablet state changed. Reason: New leader 62e99f129ff34c249544f483d0feb183. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.956784 4303 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.958704 4301 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: TxnStatusTablet state changed. Reason: New leader 62e99f129ff34c249544f483d0feb183. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.958985 4301 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.959841 4323 mvcc.cc:205] Tried to move back new op lower bound from 7315769519868264448 to 7315769519738732544. Current Snapshot: MvccSnapshot[applied={T|T < 7315769519868264448}]
I20260807 04:22:22.962152 4318 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.962432 4318 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:22:22.963261 4302 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.963519 4302 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 62e99f129ff34c249544f483d0feb183: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:22:22.966940 4324 mvcc.cc:205] Tried to move back new op lower bound from 7315769519868264448 to 7315769519738732544. Current Snapshot: MvccSnapshot[applied={T|T < 7315769519868264448 or (T in {7315769519868264448})}]
I20260807 04:22:22.967658 4301 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.967886 4301 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P b5f30e6489f6443b9e271db87cb5fd14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.970568 4303 tablet_replica.cc:439] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } } }
I20260807 04:22:22.970850 4303 tablet_replica.cc:442] T 80a0875826064699948060b463caa50b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:22:22.979099 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.973593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.1.244.193:37553)
0807 04:22:22.973806 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.973812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.974049 (+ 237us) server_negotiation.cc:407] Connection header received
0807 04:22:22.974278 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.974283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.974370 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.974517 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:22.974880 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.975639 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.977002 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.977398 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.977523 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.977979 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:22.978506 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:22.978653 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.978732 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.978814 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 04:22:22.995450 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.989714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44607 (local address 127.1.244.193:59967)
0807 04:22:22.990316 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:22.990331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:22.990448 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:22.990755 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:22.990758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:22.990766 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:22.991039 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.991046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.992402 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.992405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.993469 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:22.993480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.994446 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.994450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:22.994561 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.994606 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:22.995019 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:22.995026 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:22:22.995080 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 04:22:22.995173 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":303,"threads_started":1}
I20260807 04:22:22.995509 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:22.989832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:59967 (local address 127.1.244.195:44607)
0807 04:22:22.990040 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:22.990050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:22.990425 (+ 375us) server_negotiation.cc:407] Connection header received
0807 04:22:22.990559 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:22.990565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:22.990648 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:22.990745 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:22.991169 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.992290 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.993604 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:22.994290 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:22.994422 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:22.994796 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:22.994919 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:22.995003 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:22.995127 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 04:22:22.995210 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:22:23.005024 4085 raft_consensus.cc:1261] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Refusing update from remote peer 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:23.005354 4229 raft_consensus.cc:1261] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Refusing update from remote peer 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:23.005744 4306 consensus_queue.cc:1035] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:23.006175 4303 consensus_queue.cc:1035] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:23.043111 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:23.036381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:37553 (local address 127.1.244.195:56715)
0807 04:22:23.037091 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:23.037111 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:23.037261 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:23.037696 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:23.037701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:23.037713 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:23.038106 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:23.038120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.039401 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.039407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:23.040915 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:23.040927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.041852 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.041856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:23.042043 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:23.042110 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:23.042613 (+ 503us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:23.042623 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:22:23.042685 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 04:22:23.042780 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":433,"threads_started":1}
I20260807 04:22:23.043102 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:23.036303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56715 (local address 127.1.244.193:37553)
0807 04:22:23.036464 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:23.036499 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:23.037186 (+ 687us) server_negotiation.cc:407] Connection header received
0807 04:22:23.037489 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:23.037494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:23.037566 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:23.037684 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:23.038271 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.039258 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.041058 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.041592 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.041707 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:23.042351 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:23.042488 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:23.042580 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:23.042727 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 04:22:23.042809 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 04:22:23.082099 3783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:23.082438 3783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:23.086512 3783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Sending DeleteTablet for 3 replicas of tablet ea936869683448c98dcf75973ae0303b
I20260807 04:22:23.087603 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet ea936869683448c98dcf75973ae0303b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:23 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33154
I20260807 04:22:23.087709 4065 tablet_service.cc:1514] Processing DeleteTablet for tablet ea936869683448c98dcf75973ae0303b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:23 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37402
I20260807 04:22:23.090426 4205 tablet_service.cc:1514] Processing DeleteTablet for tablet ea936869683448c98dcf75973ae0303b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:23 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47374
I20260807 04:22:23.095381 3783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:23.096282 3783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:23.103041 4343 tablet_replica.cc:330] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: stopping tablet replica
I20260807 04:22:23.103574 4343 raft_consensus.cc:2229] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:23.104739 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7712c1b8724f28b38cb1e336e7ad2b (DEFAULT_TABLE table=test-table [id=2db8c19fd2c44279a881ac589c40c3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:23.105315 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7712c1b8724f28b38cb1e336e7ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:23.105579 4064 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7712c1b8724f28b38cb1e336e7ad2b (DEFAULT_TABLE table=test-table [id=2db8c19fd2c44279a881ac589c40c3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:23.106077 4064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7712c1b8724f28b38cb1e336e7ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:23.108702 4204 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7712c1b8724f28b38cb1e336e7ad2b (DEFAULT_TABLE table=test-table [id=2db8c19fd2c44279a881ac589c40c3c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:23.109242 4204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7712c1b8724f28b38cb1e336e7ad2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:23.116362 4343 raft_consensus.cc:2258] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:23.119988 4345 tablet_replica.cc:330] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: stopping tablet replica
I20260807 04:22:23.120973 4296 tablet_bootstrap.cc:492] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Bootstrap starting.
I20260807 04:22:23.121594 4345 raft_consensus.cc:2229] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:23.122088 4345 raft_consensus.cc:2258] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:23.122816 4296 tablet_bootstrap.cc:654] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:23.123340 4297 tablet_bootstrap.cc:492] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Bootstrap starting.
I20260807 04:22:23.124421 4345 ts_tablet_manager.cc:1905] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:23.125442 4297 tablet_bootstrap.cc:654] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:23.127526 4344 tablet_replica.cc:330] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: stopping tablet replica
I20260807 04:22:23.127983 4344 raft_consensus.cc:2229] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:23.128063 4343 ts_tablet_manager.cc:1905] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:23.128382 4344 raft_consensus.cc:2258] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:23.132742 4295 tablet_bootstrap.cc:492] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Bootstrap starting.
I20260807 04:22:23.133695 4344 ts_tablet_manager.cc:1905] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:23.134931 4295 tablet_bootstrap.cc:654] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:23.137415 4297 tablet_bootstrap.cc:492] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: No bootstrap required, opened a new log
I20260807 04:22:23.137600 4297 ts_tablet_manager.cc:1397] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260807 04:22:23.138446 4297 raft_consensus.cc:348] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.138777 4297 raft_consensus.cc:374] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:23.138877 4297 raft_consensus.cc:729] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f30e6489f6443b9e271db87cb5fd14, State: Initialized, Role: FOLLOWER
I20260807 04:22:23.139236 4297 consensus_queue.cc:260] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.140848 4297 ts_tablet_manager.cc:1428] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:23.143366 4345 ts_tablet_manager.cc:1918] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:23.143556 4345 log.cc:1196] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/ea936869683448c98dcf75973ae0303b
I20260807 04:22:23.144170 4345 ts_tablet_manager.cc:1939] T ea936869683448c98dcf75973ae0303b P 62e99f129ff34c249544f483d0feb183: Deleting consensus metadata
I20260807 04:22:23.146021 3768 catalog_manager.cc:4841] TS 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607): tablet ea936869683448c98dcf75973ae0303b (table test-table [id=040d6e2b68f44d51b29a29e6e3b0a96b]) successfully deleted
I20260807 04:22:23.148351 4295 tablet_bootstrap.cc:492] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: No bootstrap required, opened a new log
I20260807 04:22:23.148561 4295 ts_tablet_manager.cc:1397] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.004s
I20260807 04:22:23.149541 4295 raft_consensus.cc:348] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.149855 4295 raft_consensus.cc:374] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:23.149948 4295 raft_consensus.cc:729] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Initialized, Role: FOLLOWER
I20260807 04:22:23.150279 4295 consensus_queue.cc:260] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.151526 4344 ts_tablet_manager.cc:1918] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:23.151695 4344 log.cc:1196] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/ea936869683448c98dcf75973ae0303b
I20260807 04:22:23.152287 4344 ts_tablet_manager.cc:1939] T ea936869683448c98dcf75973ae0303b P b5f30e6489f6443b9e271db87cb5fd14: Deleting consensus metadata
I20260807 04:22:23.154161 4295 ts_tablet_manager.cc:1428] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 04:22:23.154224 3767 catalog_manager.cc:4841] TS b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099): tablet ea936869683448c98dcf75973ae0303b (table test-table [id=040d6e2b68f44d51b29a29e6e3b0a96b]) successfully deleted
I20260807 04:22:23.159555 4343 ts_tablet_manager.cc:1918] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:23.159734 4343 log.cc:1196] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/ea936869683448c98dcf75973ae0303b
I20260807 04:22:23.160964 4343 ts_tablet_manager.cc:1939] T ea936869683448c98dcf75973ae0303b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting consensus metadata
I20260807 04:22:23.161615 4296 tablet_bootstrap.cc:492] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: No bootstrap required, opened a new log
I20260807 04:22:23.161788 4296 ts_tablet_manager.cc:1397] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent bootstrapping tablet: real 0.041s user 0.019s sys 0.003s
I20260807 04:22:23.162585 3770 catalog_manager.cc:4841] TS 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553): tablet ea936869683448c98dcf75973ae0303b (table test-table [id=040d6e2b68f44d51b29a29e6e3b0a96b]) successfully deleted
I20260807 04:22:23.162560 4296 raft_consensus.cc:348] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.162809 4296 raft_consensus.cc:374] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:23.162921 4296 raft_consensus.cc:729] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Initialized, Role: FOLLOWER
I20260807 04:22:23.163178 4296 consensus_queue.cc:260] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.164263 4296 ts_tablet_manager.cc:1428] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 04:22:23.166431 4302 raft_consensus.cc:482] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:23.166579 4302 raft_consensus.cc:504] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.167268 4302 leader_election.cc:288] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553), b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099)
I20260807 04:22:23.167850 4084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7712c1b8724f28b38cb1e336e7ad2b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14" is_pre_election: true
I20260807 04:22:23.167948 3950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7712c1b8724f28b38cb1e336e7ad2b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" is_pre_election: true
I20260807 04:22:23.168161 3950 raft_consensus.cc:2454] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 0.
I20260807 04:22:23.168165 4084 raft_consensus.cc:2454] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 0.
I20260807 04:22:23.168627 4159 leader_election.cc:302] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4; no voters:
I20260807 04:22:23.169209 4302 raft_consensus.cc:2781] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:23.169332 4302 raft_consensus.cc:482] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:23.169440 4302 raft_consensus.cc:3037] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:23.172560 4302 raft_consensus.cc:504] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.173168 4302 leader_election.cc:288] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 election: Requested vote from peers 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553), b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099)
I20260807 04:22:23.173794 3950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7712c1b8724f28b38cb1e336e7ad2b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4"
I20260807 04:22:23.173987 3950 raft_consensus.cc:3037] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:23.175134 4084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7712c1b8724f28b38cb1e336e7ad2b" candidate_uuid: "62e99f129ff34c249544f483d0feb183" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14"
I20260807 04:22:23.175333 4084 raft_consensus.cc:3037] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:23.177183 3950 raft_consensus.cc:2454] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 1.
I20260807 04:22:23.177634 4159 leader_election.cc:302] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4; no voters:
I20260807 04:22:23.177984 4302 raft_consensus.cc:2781] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:23.178133 4302 raft_consensus.cc:686] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 1 LEADER]: Becoming Leader. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Running, Role: LEADER
I20260807 04:22:23.178395 4302 consensus_queue.cc:237] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:23.179174 4084 raft_consensus.cc:2454] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62e99f129ff34c249544f483d0feb183 in term 1.
I20260807 04:22:23.189852 3783 catalog_manager.cc:5495] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62e99f129ff34c249544f483d0feb183 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "62e99f129ff34c249544f483d0feb183" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } health_report { overall_health: HEALTHY } } }
I20260807 04:22:23.228142 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:23.221642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47392 (local address 127.1.244.195:44607)
0807 04:22:23.221798 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:23.221804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:23.221823 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:22:23.222009 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:23.222014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:23.222094 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:23.222184 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:23.222625 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.223930 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.225820 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.226545 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.226681 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:23.227213 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:23.227390 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:23.227492 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:23.227690 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 04:22:23.227784 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:22:23.244789 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:23.238320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:43301 (local address 127.1.244.195:44607)
0807 04:22:23.238799 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:23.238805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:23.238820 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:22:23.238868 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:23.238872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:23.238944 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:23.239029 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:23.239462 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.240674 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.242615 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.243316 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.243452 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:23.243951 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:23.244085 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:23.244176 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:23.244381 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 04:22:23.244463 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365}
I20260807 04:22:23.244843 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:23.238179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:44607 (local address 127.1.244.195:43301)
0807 04:22:23.238456 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:23.238476 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:23.238582 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:23.239076 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:23.239081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:23.239098 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:22:23.239318 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:23.239329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.240829 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.240835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:23.242454 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:23.242469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:23.243479 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:23.243485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:23.243712 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:23.243765 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:22:23.244196 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:22:23.244211 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 04:22:23.244313 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 04:22:23.244473 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 04:22:23.255309 3950 raft_consensus.cc:1261] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Refusing update from remote peer 62e99f129ff34c249544f483d0feb183: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:23.255347 4085 raft_consensus.cc:1261] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Refusing update from remote peer 62e99f129ff34c249544f483d0feb183: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:23.255920 4335 consensus_queue.cc:1035] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:23.256713 4318 consensus_queue.cc:1035] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:22:23.292678 4323 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 04:22:24.891294 3783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:24.891543 3783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:24.895323 3783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Sending DeleteTablet for 3 replicas of tablet 9e7712c1b8724f28b38cb1e336e7ad2b
I20260807 04:22:24.896031 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7712c1b8724f28b38cb1e336e7ad2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:24 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33154
I20260807 04:22:24.896173 4064 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7712c1b8724f28b38cb1e336e7ad2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:24 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37402
I20260807 04:22:24.896914 4373 tablet_replica.cc:330] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: stopping tablet replica
I20260807 04:22:24.897214 4373 raft_consensus.cc:2229] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:24.897536 4374 tablet_replica.cc:330] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: stopping tablet replica
I20260807 04:22:24.897536 4373 raft_consensus.cc:2258] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:24.897727 4374 raft_consensus.cc:2229] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:24.897971 4374 raft_consensus.cc:2258] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:24.898231 4205 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7712c1b8724f28b38cb1e336e7ad2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:24 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47374
I20260807 04:22:24.903569 4375 tablet_replica.cc:330] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: stopping tablet replica
I20260807 04:22:24.903796 4375 raft_consensus.cc:2229] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:24.904374 4373 ts_tablet_manager.cc:1905] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:24.904464 4375 raft_consensus.cc:2258] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:24.905303 4374 ts_tablet_manager.cc:1905] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:24.906234 4375 ts_tablet_manager.cc:1905] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:24.911980 4374 ts_tablet_manager.cc:1918] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:22:24.912122 4374 log.cc:1196] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/9e7712c1b8724f28b38cb1e336e7ad2b
I20260807 04:22:24.912616 4374 ts_tablet_manager.cc:1939] T 9e7712c1b8724f28b38cb1e336e7ad2b P b5f30e6489f6443b9e271db87cb5fd14: Deleting consensus metadata
I20260807 04:22:24.912590 4373 ts_tablet_manager.cc:1918] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:22:24.912758 4375 ts_tablet_manager.cc:1918] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:22:24.912801 4373 log.cc:1196] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/9e7712c1b8724f28b38cb1e336e7ad2b
I20260807 04:22:24.912894 4375 log.cc:1196] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/9e7712c1b8724f28b38cb1e336e7ad2b
I20260807 04:22:24.913357 4375 ts_tablet_manager.cc:1939] T 9e7712c1b8724f28b38cb1e336e7ad2b P 62e99f129ff34c249544f483d0feb183: Deleting consensus metadata
I20260807 04:22:24.913367 4373 ts_tablet_manager.cc:1939] T 9e7712c1b8724f28b38cb1e336e7ad2b P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting consensus metadata
I20260807 04:22:24.913911 3767 catalog_manager.cc:4841] TS b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099): tablet 9e7712c1b8724f28b38cb1e336e7ad2b (table test-table [id=2db8c19fd2c44279a881ac589c40c3c9]) successfully deleted
I20260807 04:22:24.914631 3768 catalog_manager.cc:4841] TS 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607): tablet 9e7712c1b8724f28b38cb1e336e7ad2b (table test-table [id=2db8c19fd2c44279a881ac589c40c3c9]) successfully deleted
I20260807 04:22:24.914673 3770 catalog_manager.cc:4841] TS 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553): tablet 9e7712c1b8724f28b38cb1e336e7ad2b (table test-table [id=2db8c19fd2c44279a881ac589c40c3c9]) successfully deleted
Aug 07 04:22:24 dist-test-slave-zkpd krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:22:24 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:24.951274 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:24.937434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.1.244.254:42571)
0807 04:22:24.938008 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:24.938014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:24.938040 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:22:24.938179 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:24.938183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:24.938245 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:24.938338 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:24.939699 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:24.940509 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:24.941676 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:24.942083 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:24.942172 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:24.945888 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:24.945893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:24.945896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:24.945930 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:24.948755 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:24.949530 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:24.949534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:24.949536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:24.949690 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:24.950028 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:24.950031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:24.950033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:24.950275 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:24.950443 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:24.950793 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 04:22:24.950956 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":261,"threads_started":1}
I20260807 04:22:24.954697 3783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:24.955330 3783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:24.962003 3924 tablet_service.cc:1467] Processing CreateTablet for tablet da7cddb2af614439831e016a694c0d5c (DEFAULT_TABLE table=test-table [id=6e68ab789ab64ca3a61b964078c29c96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:24.962105 4064 tablet_service.cc:1467] Processing CreateTablet for tablet da7cddb2af614439831e016a694c0d5c (DEFAULT_TABLE table=test-table [id=6e68ab789ab64ca3a61b964078c29c96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:24.962453 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7cddb2af614439831e016a694c0d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:24.962334 4205 tablet_service.cc:1467] Processing CreateTablet for tablet da7cddb2af614439831e016a694c0d5c (DEFAULT_TABLE table=test-table [id=6e68ab789ab64ca3a61b964078c29c96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:24.962563 4064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7cddb2af614439831e016a694c0d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:24.962759 4205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da7cddb2af614439831e016a694c0d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:24.969447 4385 tablet_bootstrap.cc:492] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Bootstrap starting.
I20260807 04:22:24.971549 4385 tablet_bootstrap.cc:654] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:24.974411 4385 tablet_bootstrap.cc:492] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: No bootstrap required, opened a new log
I20260807 04:22:24.974493 4386 tablet_bootstrap.cc:492] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Bootstrap starting.
I20260807 04:22:24.974674 4385 ts_tablet_manager.cc:1397] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 04:22:24.975703 4385 raft_consensus.cc:348] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:24.976076 4385 raft_consensus.cc:374] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:24.976224 4385 raft_consensus.cc:729] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Initialized, Role: FOLLOWER
I20260807 04:22:24.976325 4386 tablet_bootstrap.cc:654] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:24.976557 4385 consensus_queue.cc:260] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:24.977649 4385 ts_tablet_manager.cc:1428] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 04:22:24.978116 4384 tablet_bootstrap.cc:492] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Bootstrap starting.
I20260807 04:22:24.979274 4386 tablet_bootstrap.cc:492] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: No bootstrap required, opened a new log
I20260807 04:22:24.979420 4386 ts_tablet_manager.cc:1397] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 04:22:24.979825 4384 tablet_bootstrap.cc:654] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:24.980362 4386 raft_consensus.cc:348] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:24.980639 4386 raft_consensus.cc:374] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:24.980754 4386 raft_consensus.cc:729] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Initialized, Role: FOLLOWER
I20260807 04:22:24.981021 4386 consensus_queue.cc:260] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:24.982357 4386 ts_tablet_manager.cc:1428] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:24.983122 4384 tablet_bootstrap.cc:492] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: No bootstrap required, opened a new log
I20260807 04:22:24.983273 4384 ts_tablet_manager.cc:1397] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 04:22:24.983996 4384 raft_consensus.cc:348] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:24.984265 4384 raft_consensus.cc:374] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:24.984380 4384 raft_consensus.cc:729] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f30e6489f6443b9e271db87cb5fd14, State: Initialized, Role: FOLLOWER
I20260807 04:22:24.984658 4384 consensus_queue.cc:260] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:24.985661 4384 ts_tablet_manager.cc:1428] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 04:22:25.025846 4388 raft_consensus.cc:482] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:25.026051 4388 raft_consensus.cc:504] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.026687 4388 leader_election.cc:288] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099), 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:25.027199 4084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7cddb2af614439831e016a694c0d5c" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14" is_pre_election: true
I20260807 04:22:25.027279 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7cddb2af614439831e016a694c0d5c" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e99f129ff34c249544f483d0feb183" is_pre_election: true
I20260807 04:22:25.027513 4084 raft_consensus.cc:2454] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 0.
I20260807 04:22:25.027572 4228 raft_consensus.cc:2454] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 0.
I20260807 04:22:25.028064 3874 leader_election.cc:302] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4, b5f30e6489f6443b9e271db87cb5fd14; no voters:
I20260807 04:22:25.028357 4388 raft_consensus.cc:2781] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:25.028494 4388 raft_consensus.cc:482] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:25.028604 4388 raft_consensus.cc:3037] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:25.030671 4388 raft_consensus.cc:504] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.031175 4388 leader_election.cc:288] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 election: Requested vote from peers b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099), 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:25.031672 4084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7cddb2af614439831e016a694c0d5c" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14"
I20260807 04:22:25.031698 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da7cddb2af614439831e016a694c0d5c" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e99f129ff34c249544f483d0feb183"
I20260807 04:22:25.031893 4228 raft_consensus.cc:3037] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:25.031960 4084 raft_consensus.cc:3037] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:25.034901 4084 raft_consensus.cc:2454] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 1.
I20260807 04:22:25.034937 4228 raft_consensus.cc:2454] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 1.
I20260807 04:22:25.035321 3874 leader_election.cc:302] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4; no voters:
I20260807 04:22:25.035686 4388 raft_consensus.cc:2781] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:25.036131 4388 raft_consensus.cc:686] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Running, Role: LEADER
I20260807 04:22:25.036397 4388 consensus_queue.cc:237] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.038854 3782 catalog_manager.cc:5495] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } health_report { overall_health: UNKNOWN } } }
I20260807 04:22:25.077311 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:25.072555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.1.244.193:37553)
0807 04:22:25.073078 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:25.073082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:25.073101 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:22:25.073150 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:25.073154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:25.073214 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:25.073296 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:25.073624 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:25.074390 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:25.075628 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:25.076065 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:25.076166 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:25.076586 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:25.076684 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:25.076752 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:25.076876 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 04:22:25.076941 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":294,"threads_started":1}
I20260807 04:22:25.085628 4228 raft_consensus.cc:1261] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Refusing update from remote peer 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:25.085721 4084 raft_consensus.cc:1261] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Refusing update from remote peer 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:25.086266 4392 consensus_queue.cc:1035] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:25.086577 4388 consensus_queue.cc:1035] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:25.129070 3783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:25.129323 3783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:25.131991 3783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Sending DeleteTablet for 3 replicas of tablet da7cddb2af614439831e016a694c0d5c
I20260807 04:22:25.132654 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet da7cddb2af614439831e016a694c0d5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33154
I20260807 04:22:25.132805 4064 tablet_service.cc:1514] Processing DeleteTablet for tablet da7cddb2af614439831e016a694c0d5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37402
I20260807 04:22:25.132882 4205 tablet_service.cc:1514] Processing DeleteTablet for tablet da7cddb2af614439831e016a694c0d5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47374
I20260807 04:22:25.133085 4373 tablet_replica.cc:330] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: stopping tablet replica
I20260807 04:22:25.133083 4374 tablet_replica.cc:330] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: stopping tablet replica
I20260807 04:22:25.133155 4375 tablet_replica.cc:330] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: stopping tablet replica
I20260807 04:22:25.133278 4373 raft_consensus.cc:2229] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:25.133278 4374 raft_consensus.cc:2229] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:25.133355 4375 raft_consensus.cc:2229] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:25.133460 4374 raft_consensus.cc:2258] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:25.133538 4375 raft_consensus.cc:2258] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:25.133579 4373 raft_consensus.cc:2258] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:25.133837 3783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:25.134546 3783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:25.134634 4374 ts_tablet_manager.cc:1905] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:25.134742 4375 ts_tablet_manager.cc:1905] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:25.134891 4373 ts_tablet_manager.cc:1905] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:25.140103 4375 ts_tablet_manager.cc:1918] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:25.140281 4375 log.cc:1196] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/da7cddb2af614439831e016a694c0d5c
I20260807 04:22:25.140744 4375 ts_tablet_manager.cc:1939] T da7cddb2af614439831e016a694c0d5c P 62e99f129ff34c249544f483d0feb183: Deleting consensus metadata
I20260807 04:22:25.141847 3768 catalog_manager.cc:4841] TS 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607): tablet da7cddb2af614439831e016a694c0d5c (table test-table [id=6e68ab789ab64ca3a61b964078c29c96]) successfully deleted
I20260807 04:22:25.142509 4374 ts_tablet_manager.cc:1918] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:25.142668 4374 log.cc:1196] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/da7cddb2af614439831e016a694c0d5c
I20260807 04:22:25.142695 4373 ts_tablet_manager.cc:1918] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:22:25.142835 4373 log.cc:1196] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/da7cddb2af614439831e016a694c0d5c
I20260807 04:22:25.143107 4374 ts_tablet_manager.cc:1939] T da7cddb2af614439831e016a694c0d5c P b5f30e6489f6443b9e271db87cb5fd14: Deleting consensus metadata
I20260807 04:22:25.143260 4373 ts_tablet_manager.cc:1939] T da7cddb2af614439831e016a694c0d5c P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting consensus metadata
I20260807 04:22:25.144174 3767 catalog_manager.cc:4841] TS b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099): tablet da7cddb2af614439831e016a694c0d5c (table test-table [id=6e68ab789ab64ca3a61b964078c29c96]) successfully deleted
I20260807 04:22:25.144281 3770 catalog_manager.cc:4841] TS 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553): tablet da7cddb2af614439831e016a694c0d5c (table test-table [id=6e68ab789ab64ca3a61b964078c29c96]) successfully deleted
I20260807 04:22:25.145454 3924 tablet_service.cc:1467] Processing CreateTablet for tablet 4867da2888e54a86ac8be75a892cf2f8 (DEFAULT_TABLE table=test-table [id=3d1f654c2ed34c319f75a407bd88d3bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:25.145876 3924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4867da2888e54a86ac8be75a892cf2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:25.146148 4205 tablet_service.cc:1467] Processing CreateTablet for tablet 4867da2888e54a86ac8be75a892cf2f8 (DEFAULT_TABLE table=test-table [id=3d1f654c2ed34c319f75a407bd88d3bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:25.146484 4205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4867da2888e54a86ac8be75a892cf2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:25.150593 4385 tablet_bootstrap.cc:492] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Bootstrap starting.
I20260807 04:22:25.151149 4386 tablet_bootstrap.cc:492] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Bootstrap starting.
I20260807 04:22:25.151103 4064 tablet_service.cc:1467] Processing CreateTablet for tablet 4867da2888e54a86ac8be75a892cf2f8 (DEFAULT_TABLE table=test-table [id=3d1f654c2ed34c319f75a407bd88d3bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:25.151520 4064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4867da2888e54a86ac8be75a892cf2f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:25.152309 4385 tablet_bootstrap.cc:654] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:25.152451 4386 tablet_bootstrap.cc:654] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:25.154219 4386 tablet_bootstrap.cc:492] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: No bootstrap required, opened a new log
I20260807 04:22:25.154384 4386 ts_tablet_manager.cc:1397] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 04:22:25.154948 4386 raft_consensus.cc:348] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.155160 4386 raft_consensus.cc:374] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:25.155256 4386 raft_consensus.cc:729] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e99f129ff34c249544f483d0feb183, State: Initialized, Role: FOLLOWER
I20260807 04:22:25.155547 4386 consensus_queue.cc:260] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.156060 4386 ts_tablet_manager.cc:1428] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 04:22:25.159057 4384 tablet_bootstrap.cc:492] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Bootstrap starting.
I20260807 04:22:25.160427 4384 tablet_bootstrap.cc:654] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:25.162153 4385 tablet_bootstrap.cc:492] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: No bootstrap required, opened a new log
I20260807 04:22:25.162331 4385 ts_tablet_manager.cc:1397] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260807 04:22:25.162425 4384 tablet_bootstrap.cc:492] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: No bootstrap required, opened a new log
I20260807 04:22:25.162582 4384 ts_tablet_manager.cc:1397] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 04:22:25.162938 4385 raft_consensus.cc:348] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.163157 4385 raft_consensus.cc:374] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:25.163254 4385 raft_consensus.cc:729] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Initialized, Role: FOLLOWER
I20260807 04:22:25.163106 4384 raft_consensus.cc:348] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.163369 4384 raft_consensus.cc:374] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:25.163470 4384 raft_consensus.cc:729] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5f30e6489f6443b9e271db87cb5fd14, State: Initialized, Role: FOLLOWER
I20260807 04:22:25.163508 4385 consensus_queue.cc:260] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.163700 4384 consensus_queue.cc:260] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.164085 4385 ts_tablet_manager.cc:1428] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 04:22:25.164741 4384 ts_tablet_manager.cc:1428] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:22:25.167796 4399 raft_consensus.cc:482] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:25.167981 4399 raft_consensus.cc:504] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.168563 4399 leader_election.cc:288] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099), 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:25.168968 4084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867da2888e54a86ac8be75a892cf2f8" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14" is_pre_election: true
I20260807 04:22:25.168999 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867da2888e54a86ac8be75a892cf2f8" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e99f129ff34c249544f483d0feb183" is_pre_election: true
I20260807 04:22:25.169174 4084 raft_consensus.cc:2454] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 0.
I20260807 04:22:25.169210 4228 raft_consensus.cc:2454] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 0.
I20260807 04:22:25.169543 3873 leader_election.cc:302] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b3cb2cae7ee4faf9498eec17d0f7ac4, b5f30e6489f6443b9e271db87cb5fd14; no voters:
I20260807 04:22:25.169889 4399 raft_consensus.cc:2781] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:25.170006 4399 raft_consensus.cc:482] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:25.170094 4399 raft_consensus.cc:3037] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:25.172353 4399 raft_consensus.cc:504] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.172823 4399 leader_election.cc:288] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 election: Requested vote from peers b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099), 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607)
I20260807 04:22:25.173246 4228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867da2888e54a86ac8be75a892cf2f8" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62e99f129ff34c249544f483d0feb183"
I20260807 04:22:25.173249 4084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4867da2888e54a86ac8be75a892cf2f8" candidate_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5f30e6489f6443b9e271db87cb5fd14"
I20260807 04:22:25.173419 4228 raft_consensus.cc:3037] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:25.173447 4084 raft_consensus.cc:3037] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:25.175302 4084 raft_consensus.cc:2454] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 1.
I20260807 04:22:25.175309 4228 raft_consensus.cc:2454] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b3cb2cae7ee4faf9498eec17d0f7ac4 in term 1.
I20260807 04:22:25.175696 3874 leader_election.cc:302] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62e99f129ff34c249544f483d0feb183, 7b3cb2cae7ee4faf9498eec17d0f7ac4; no voters:
I20260807 04:22:25.176067 4399 raft_consensus.cc:2781] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:25.176211 4399 raft_consensus.cc:686] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 7b3cb2cae7ee4faf9498eec17d0f7ac4, State: Running, Role: LEADER
I20260807 04:22:25.176476 4399 consensus_queue.cc:237] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } }
I20260807 04:22:25.178982 3784 catalog_manager.cc:5495] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b3cb2cae7ee4faf9498eec17d0f7ac4" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 37553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 } health_report { overall_health: UNKNOWN } } }
I20260807 04:22:25.206086 4085 raft_consensus.cc:1261] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Refusing update from remote peer 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:25.206184 4228 raft_consensus.cc:1261] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Refusing update from remote peer 7b3cb2cae7ee4faf9498eec17d0f7ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:25.206663 4399 consensus_queue.cc:1035] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5f30e6489f6443b9e271db87cb5fd14" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:25.206984 4392 consensus_queue.cc:1035] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62e99f129ff34c249544f483d0feb183" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 44607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:22:25.225858 4358 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 04:22:26.804028 3783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:26.804286 3783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:26.808761 3783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b27487f08a043ae9d02ae4bac138482: Sending DeleteTablet for 3 replicas of tablet 4867da2888e54a86ac8be75a892cf2f8
I20260807 04:22:26.809504 3924 tablet_service.cc:1514] Processing DeleteTablet for tablet 4867da2888e54a86ac8be75a892cf2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:26 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33154
I20260807 04:22:26.809528 4064 tablet_service.cc:1514] Processing DeleteTablet for tablet 4867da2888e54a86ac8be75a892cf2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:26 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:37402
I20260807 04:22:26.809968 4205 tablet_service.cc:1514] Processing DeleteTablet for tablet 4867da2888e54a86ac8be75a892cf2f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:26 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:47374
I20260807 04:22:26.810163 4430 tablet_replica.cc:330] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: stopping tablet replica
I20260807 04:22:26.810283 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3857
I20260807 04:22:26.810456 4430 raft_consensus.cc:2229] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:26.810873 4430 raft_consensus.cc:2258] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:26.812415 4431 tablet_replica.cc:330] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: stopping tablet replica
I20260807 04:22:26.812656 4431 raft_consensus.cc:2229] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:26.812722 4430 ts_tablet_manager.cc:1905] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:26.812914 4431 raft_consensus.cc:2258] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:26.820765 4431 ts_tablet_manager.cc:1905] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:26.821321 4432 tablet_replica.cc:330] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: stopping tablet replica
I20260807 04:22:26.821621 4432 raft_consensus.cc:2229] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:26.821879 4432 raft_consensus.cc:2258] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:26.822185 4430 ts_tablet_manager.cc:1918] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:22:26.822606 4430 log.cc:1196] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/4867da2888e54a86ac8be75a892cf2f8
I20260807 04:22:26.823380 4430 ts_tablet_manager.cc:1939] T 4867da2888e54a86ac8be75a892cf2f8 P 7b3cb2cae7ee4faf9498eec17d0f7ac4: Deleting consensus metadata
I20260807 04:22:26.823844 4432 ts_tablet_manager.cc:1905] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:26.824498 3770 catalog_manager.cc:4841] TS 7b3cb2cae7ee4faf9498eec17d0f7ac4 (127.1.244.193:37553): tablet 4867da2888e54a86ac8be75a892cf2f8 (table test-table [id=3d1f654c2ed34c319f75a407bd88d3bb]) successfully deleted
I20260807 04:22:26.826413 4431 ts_tablet_manager.cc:1918] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:22:26.826575 4431 log.cc:1196] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/4867da2888e54a86ac8be75a892cf2f8
I20260807 04:22:26.826951 4431 ts_tablet_manager.cc:1939] T 4867da2888e54a86ac8be75a892cf2f8 P b5f30e6489f6443b9e271db87cb5fd14: Deleting consensus metadata
I20260807 04:22:26.828140 3767 catalog_manager.cc:4841] TS b5f30e6489f6443b9e271db87cb5fd14 (127.1.244.194:42099): tablet 4867da2888e54a86ac8be75a892cf2f8 (table test-table [id=3d1f654c2ed34c319f75a407bd88d3bb]) successfully deleted
I20260807 04:22:26.830157 4432 ts_tablet_manager.cc:1918] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 04:22:26.830302 4432 log.cc:1196] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/4867da2888e54a86ac8be75a892cf2f8
I20260807 04:22:26.830786 4432 ts_tablet_manager.cc:1939] T 4867da2888e54a86ac8be75a892cf2f8 P 62e99f129ff34c249544f483d0feb183: Deleting consensus metadata
I20260807 04:22:26.832028 3768 catalog_manager.cc:4841] TS 62e99f129ff34c249544f483d0feb183 (127.1.244.195:44607): tablet 4867da2888e54a86ac8be75a892cf2f8 (table test-table [id=3d1f654c2ed34c319f75a407bd88d3bb]) successfully deleted
Aug 07 04:22:26 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:26.844967 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:26.811594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.1.244.193:37553)
0807 04:22:26.822382 (+ 10788us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:26.822388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:26.828365 (+ 5977us) server_negotiation.cc:407] Connection header received
0807 04:22:26.828437 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:26.828444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:26.828526 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:26.828657 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:26.830317 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:26.831557 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:26.833485 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:26.834148 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:26.834236 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:26.840009 (+ 5773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:26.840024 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:26.840040 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:26.840071 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:26.842600 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:26.843183 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:26.843189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:26.843191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:26.843303 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:26.843650 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:26.843653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:26.843655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:26.844029 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:26.844249 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:26.844451 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 04:22:26.844611 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10666,"thread_start_us":10549,"threads_started":1}
W20260807 04:22:26.845628 3990 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33174
W20260807 04:22:26.846169 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3857: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:26.846395 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3857
I20260807 04:22:26.871378 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3999
Aug 07 04:22:26 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:26.885830 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:26.872185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37418 (local address 127.1.244.194:42099)
0807 04:22:26.872743 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:26.872747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:26.872762 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:22:26.872816 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:26.872820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:26.872875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:26.872966 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:26.874220 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:26.874939 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:26.876209 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:26.876609 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:26.876698 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:26.880776 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:26.880788 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:26.880801 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:26.880835 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:26.883406 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:26.884042 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:26.884051 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:26.884060 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:26.884212 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:26.884549 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:26.884552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:26.884554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:26.884875 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:26.885087 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:26.885286 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 04:22:26.885453 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":299,"threads_started":1}
W20260807 04:22:26.886508 4131 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418
W20260807 04:22:26.887136 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3999: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:26.887276 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3999
I20260807 04:22:26.907616 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4140
Aug 07 04:22:26 dist-test-slave-zkpd krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076544, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:26.922652 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:26.908397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.1.244.195:44607)
0807 04:22:26.908968 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:26.908972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:26.908988 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:22:26.909043 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:26.909046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:26.909112 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:26.909205 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:26.910467 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:26.911239 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:26.912627 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:26.913028 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:26.913121 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:26.917272 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:26.917287 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:26.917303 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:26.917333 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:26.920288 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:26.920847 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:26.920853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:26.920856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:26.920978 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:26.921336 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:26.921339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:26.921341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:26.921681 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:26.921883 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:26.922157 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 04:22:26.922377 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":296,"threads_started":1}
W20260807 04:22:26.923293 4271 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47408
W20260807 04:22:26.923856 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4140: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:26.924095 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 4140
I20260807 04:22:26.951632 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 3751
I20260807 04:22:26.953042 3825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:27.068802 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 3751
2026-08-07T04:22:27Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7977 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:27 dist-test-slave-zkpd krb5kdc[4440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:27 dist-test-slave-zkpd krb5kdc[4440](info): set up 2 sockets
Aug 07 04:22:27 dist-test-slave-zkpd krb5kdc[4440](info): commencing operation
krb5kdc: starting...
W20260807 04:22:28.182981 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:28 dist-test-slave-zkpd krb5kdc[4440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:28.389564 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45505
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:28.708096 4454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:28.708442 4454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:28.708611 4454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:28.715727 4454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:28.715860 4454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:28.715986 4454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:28.716073 4454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:28.716140 4454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:28.725821 4454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:28.727065 4454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:28.728366 4454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:28.740049 4459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:28.740095 4460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:28.740135 4462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:28.741626 4454 server_base.cc:1034] running on GCE node
I20260807 04:22:28.742352 4454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:28.743630 4454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:28.744832 4454 hybrid_clock.cc:648] HybridClock initialized: now 1786076548744755 us; error 65 us; skew 500 ppm
I20260807 04:22:28.745143 4454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:28 dist-test-slave-zkpd krb5kdc[4440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076548, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:28.751107 4454 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:28.752550 4454 webserver.cc:460] Webserver started at http://127.1.244.254:45131/ using document root <none> and password file <none>
I20260807 04:22:28.753212 4454 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:28.753311 4454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:28.753582 4454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:28.756526 4454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "fe344a5232fa4769badddcdb8bd17946"
format_stamp: "Formatted at 2026-08-07 04:22:28 on dist-test-slave-zkpd"
server_key: "e783fbf44069c78b9a622402f10672a3"
server_key_iv: "cf5a86fb929e8fd50aecc845f9daf676"
server_key_version: "encryptionkey@0"
I20260807 04:22:28.757268 4454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "fe344a5232fa4769badddcdb8bd17946"
format_stamp: "Formatted at 2026-08-07 04:22:28 on dist-test-slave-zkpd"
server_key: "e783fbf44069c78b9a622402f10672a3"
server_key_iv: "cf5a86fb929e8fd50aecc845f9daf676"
server_key_version: "encryptionkey@0"
I20260807 04:22:28.762221 4454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:28.765477 4469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:28.766712 4454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 04:22:28.766889 4454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "fe344a5232fa4769badddcdb8bd17946"
format_stamp: "Formatted at 2026-08-07 04:22:28 on dist-test-slave-zkpd"
server_key: "e783fbf44069c78b9a622402f10672a3"
server_key_iv: "cf5a86fb929e8fd50aecc845f9daf676"
server_key_version: "encryptionkey@0"
I20260807 04:22:28.767086 4454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:28.782207 4454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:28.786298 4454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:28.786586 4454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:28.805184 4454 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45505
I20260807 04:22:28.805202 4521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45505 every 8 connection(s)
I20260807 04:22:28.806573 4454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:28.807217 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 4454
I20260807 04:22:28.807672 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:28.808514 2003 external_mini_cluster.cc:1428] Setting key cda9d1de6a43eda1b0480e28db2c5889
I20260807 04:22:28.811143 4522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:22:28 dist-test-slave-zkpd krb5kdc[4440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:28.824501 4522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946: Bootstrap starting.
I20260807 04:22:28.828696 4522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:28.829947 4522 log.cc:824] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:28.833058 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:28.810489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.1.244.254:45505)
0807 04:22:28.811329 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:28.811348 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:28.811389 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:22:28.813079 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:28.813098 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:28.813481 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:28.813824 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:28.815738 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:28.817360 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:28.819150 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:28.819752 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:28.819870 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:28.824597 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:28.824609 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:28.824626 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:28.824675 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:28.828612 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:28.829256 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:28.829263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:28.829271 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:28.829475 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:28.829828 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:28.829833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:28.829836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:28.830547 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:28.830814 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:28.831343 (+ 529us) server_negotiation.cc:299] Negotiation successful
0807 04:22:28.831740 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":402,"threads_started":1}
I20260807 04:22:28.833277 4522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946: No bootstrap required, opened a new log
I20260807 04:22:28.841040 4522 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe344a5232fa4769badddcdb8bd17946" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45505 } }
I20260807 04:22:28.841354 4522 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:28.841434 4522 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe344a5232fa4769badddcdb8bd17946, State: Initialized, Role: FOLLOWER
I20260807 04:22:28.841959 4522 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe344a5232fa4769badddcdb8bd17946" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45505 } }
I20260807 04:22:28.842160 4522 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:28.842254 4522 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:28.842376 4522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:28.844436 4522 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe344a5232fa4769badddcdb8bd17946" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45505 } }
I20260807 04:22:28.844863 4522 leader_election.cc:302] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe344a5232fa4769badddcdb8bd17946; no voters:
I20260807 04:22:28.845368 4522 leader_election.cc:288] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:28.845510 4527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:28.846207 4527 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [term 1 LEADER]: Becoming Leader. State: Replica: fe344a5232fa4769badddcdb8bd17946, State: Running, Role: LEADER
I20260807 04:22:28.846761 4527 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe344a5232fa4769badddcdb8bd17946" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45505 } }
I20260807 04:22:28.847471 4522 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:28.850056 4528 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe344a5232fa4769badddcdb8bd17946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe344a5232fa4769badddcdb8bd17946" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45505 } } }
I20260807 04:22:28.850515 4528 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:28.850026 4529 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe344a5232fa4769badddcdb8bd17946. Latest consensus state: current_term: 1 leader_uuid: "fe344a5232fa4769badddcdb8bd17946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe344a5232fa4769badddcdb8bd17946" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45505 } } }
I20260807 04:22:28.850890 4529 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:28.851387 4536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:28.856447 4536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:28.865514 4536 catalog_manager.cc:1347] Generated new cluster ID: 55fe8ea6a5ca4d8cb972123bfa5cf820
I20260807 04:22:28.865623 4536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:28.875352 4536 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:28.876616 4536 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:28.888513 4536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe344a5232fa4769badddcdb8bd17946: Generated new TSK 0
I20260807 04:22:28.889331 4536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:28.939282 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45505
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:29.269148 4549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:29.269398 4549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:29.269486 4549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:29.277135 4549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:29.277232 4549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:29.277338 4549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:29.286367 4549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:29.287496 4549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:29.288865 4549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:29.302054 4554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:29.302021 4557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:29.302095 4555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:29.303360 4549 server_base.cc:1034] running on GCE node
I20260807 04:22:29.303973 4549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:29.304845 4549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:29.306054 4549 hybrid_clock.cc:648] HybridClock initialized: now 1786076549306004 us; error 33 us; skew 500 ppm
I20260807 04:22:29.306363 4549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:29 dist-test-slave-zkpd krb5kdc[4440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:29.309703 4549 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:29.311147 4549 webserver.cc:460] Webserver started at http://127.1.244.193:37965/ using document root <none> and password file <none>
I20260807 04:22:29.311830 4549 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:29.312012 4549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:29.312314 4549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:29.315093 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "353e05d8a1044afe87fefc7f2d4d389d"
format_stamp: "Formatted at 2026-08-07 04:22:29 on dist-test-slave-zkpd"
server_key: "e9ea7fcc38567415d60c1a0511a99954"
server_key_iv: "cce20618084c5e79cbac37c52db7e9ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:29.315863 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "353e05d8a1044afe87fefc7f2d4d389d"
format_stamp: "Formatted at 2026-08-07 04:22:29 on dist-test-slave-zkpd"
server_key: "e9ea7fcc38567415d60c1a0511a99954"
server_key_iv: "cce20618084c5e79cbac37c52db7e9ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:29.320924 4549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 04:22:29.324426 4564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:29.325721 4549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:22:29.325904 4549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "353e05d8a1044afe87fefc7f2d4d389d"
format_stamp: "Formatted at 2026-08-07 04:22:29 on dist-test-slave-zkpd"
server_key: "e9ea7fcc38567415d60c1a0511a99954"
server_key_iv: "cce20618084c5e79cbac37c52db7e9ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:29.326103 4549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:29.353745 4549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:29.357885 4549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:29.358160 4549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:29.359107 4549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:29.360446 4549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:29.360519 4549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:29.360687 4549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:29.360769 4549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:29.391212 4549 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:42857
I20260807 04:22:29.391256 4679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:42857 every 8 connection(s)
I20260807 04:22:29.392419 4549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:29.399755 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 4549
I20260807 04:22:29.400018 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:29.400633 2003 external_mini_cluster.cc:1428] Setting key c3c055e6127c5e3ffc26302f3b83b37e
Aug 07 04:22:29 dist-test-slave-zkpd krb5kdc[4440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:29.411840 4682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:29.394866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:42237 (local address 127.1.244.254:45505)
0807 04:22:29.395286 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:29.395291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:29.396352 (+ 1061us) server_negotiation.cc:407] Connection header received
0807 04:22:29.398046 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:29.398051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:29.398127 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:29.398219 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:29.399645 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.400463 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.402204 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.402600 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.402736 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:29.406052 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:29.406058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:29.406061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:29.406093 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:29.408846 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:29.409531 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:29.409537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:29.409541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:29.409701 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:29.410204 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:29.410207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:29.410209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:29.410474 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:29.410585 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:29.411423 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 04:22:29.411577 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":212,"threads_started":1}
I20260807 04:22:29.413339 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:29.395443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45505 (local address 127.1.244.193:42237)
0807 04:22:29.396187 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:29.396231 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:29.397804 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:29.398416 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:29.398423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:29.398926 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:29.399453 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:29.399469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.400607 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.400613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:29.402015 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:29.402032 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.402833 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.402841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:29.403076 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:29.403742 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:29.403763 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:29.405830 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:29.409000 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:29.409006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:29.409018 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:29.409415 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:29.409814 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:29.409817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:29.409819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:29.410091 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:29.410601 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:29.410611 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:29.411043 (+ 432us) client_negotiation.cc:769] Sending connection context
0807 04:22:29.411336 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 04:22:29.411872 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":251,"threads_started":1}
I20260807 04:22:29.414907 4680 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45505
I20260807 04:22:29.415199 4680 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:29.415865 4680 heartbeater.cc:511] Master 127.1.244.254:45505 requested a full tablet report, sending...
I20260807 04:22:29.418099 4487 ts_manager.cc:194] Registered new tserver with Master: 353e05d8a1044afe87fefc7f2d4d389d (127.1.244.193:42857)
I20260807 04:22:29.419504 4487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:42237
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:29.447324 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45505
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:29.798642 4686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:29.798898 4686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:29.798991 4686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:29.806223 4686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:29.806380 4686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:29.806551 4686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:29.816250 4686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:29.817353 4686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:29.818670 4686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:29.831460 4691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:29.831516 4694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:29.831542 4692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:29.832526 4686 server_base.cc:1034] running on GCE node
I20260807 04:22:29.833117 4686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:29.834002 4686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:29.835192 4686 hybrid_clock.cc:648] HybridClock initialized: now 1786076549835134 us; error 45 us; skew 500 ppm
I20260807 04:22:29.835493 4686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:29 dist-test-slave-zkpd krb5kdc[4440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:29.838650 4686 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:29.840034 4686 webserver.cc:460] Webserver started at http://127.1.244.194:44427/ using document root <none> and password file <none>
I20260807 04:22:29.840674 4686 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:29.840826 4686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:29.841123 4686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:29.843753 4686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "330cbb37fdd64b37a507f3a404612fb8"
format_stamp: "Formatted at 2026-08-07 04:22:29 on dist-test-slave-zkpd"
server_key: "d7acfc1c247cb09c96e96ffa7dc85606"
server_key_iv: "898a2185dc2536f18ac074c323fa1582"
server_key_version: "encryptionkey@0"
I20260807 04:22:29.844611 4686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "330cbb37fdd64b37a507f3a404612fb8"
format_stamp: "Formatted at 2026-08-07 04:22:29 on dist-test-slave-zkpd"
server_key: "d7acfc1c247cb09c96e96ffa7dc85606"
server_key_iv: "898a2185dc2536f18ac074c323fa1582"
server_key_version: "encryptionkey@0"
I20260807 04:22:29.849732 4686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:29.852952 4701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:29.854177 4686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:29.854347 4686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "330cbb37fdd64b37a507f3a404612fb8"
format_stamp: "Formatted at 2026-08-07 04:22:29 on dist-test-slave-zkpd"
server_key: "d7acfc1c247cb09c96e96ffa7dc85606"
server_key_iv: "898a2185dc2536f18ac074c323fa1582"
server_key_version: "encryptionkey@0"
I20260807 04:22:29.854525 4686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:29.900528 4686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:29.904788 4686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:29.905088 4686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:29.906011 4686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:29.907258 4686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:29.907342 4686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:29.907508 4686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:29.907557 4686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:29.940497 4686 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:37919
I20260807 04:22:29.940577 4817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:37919 every 8 connection(s)
I20260807 04:22:29.941766 4686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:29.946383 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 4686
I20260807 04:22:29.946579 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:29.947106 2003 external_mini_cluster.cc:1428] Setting key fd86d6360e569ab6bcc345d057e27c2c
Aug 07 04:22:29 dist-test-slave-zkpd krb5kdc[4440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076549, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:29.962782 4820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:29.944156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:34641 (local address 127.1.244.254:45505)
0807 04:22:29.944592 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:29.944597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:29.945397 (+ 800us) server_negotiation.cc:407] Connection header received
0807 04:22:29.947800 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:29.947806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:29.947971 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:29.948093 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:29.949969 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.951165 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.952674 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.953471 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.953610 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:29.957029 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:29.957034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:29.957038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:29.957068 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:29.959759 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:29.960475 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:29.960480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:29.960485 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:29.960632 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:29.961077 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:29.961081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:29.961083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:29.961337 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:29.961509 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:29.962181 (+ 672us) server_negotiation.cc:299] Negotiation successful
0807 04:22:29.962407 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":239,"threads_started":1}
I20260807 04:22:29.963732 4821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:29.944576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45505 (local address 127.1.244.194:34641)
0807 04:22:29.945239 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:29.945275 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:29.947493 (+ 2218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:29.948354 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:29.948364 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:29.949190 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:29.949794 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:29.949811 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.951313 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.951317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:29.952541 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:29.952553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:29.953674 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:29.953678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:29.953884 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:29.954539 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:29.954560 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:29.956793 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:29.959922 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:29.959930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:29.959945 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:29.960353 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:29.960746 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:29.960751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:29.960755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:29.960952 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:29.961480 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:29.961487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:29.961790 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 04:22:29.962093 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 04:22:29.962572 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":274,"threads_started":1}
I20260807 04:22:29.965263 4818 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45505
I20260807 04:22:29.965560 4818 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:29.966219 4818 heartbeater.cc:511] Master 127.1.244.254:45505 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260807 04:22:29.967670 4487 ts_manager.cc:194] Registered new tserver with Master: 330cbb37fdd64b37a507f3a404612fb8 (127.1.244.194:37919)
I20260807 04:22:29.968533 4487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:34641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:29.992239 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45505
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:30.309422 4824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:30.309727 4824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:30.309907 4824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:30.317245 4824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:30.317497 4824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:30.317679 4824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:30.327304 4824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:30.328824 4824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:30.330245 4824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:30.343601 4829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:30.343638 4830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:30.343670 4832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:30.344496 4824 server_base.cc:1034] running on GCE node
I20260807 04:22:30.345105 4824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:30.345963 4824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:30.347151 4824 hybrid_clock.cc:648] HybridClock initialized: now 1786076550347095 us; error 50 us; skew 500 ppm
I20260807 04:22:30.347608 4824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:30 dist-test-slave-zkpd krb5kdc[4440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076550, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:30.350749 4824 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:30.352116 4824 webserver.cc:460] Webserver started at http://127.1.244.195:37319/ using document root <none> and password file <none>
I20260807 04:22:30.352797 4824 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:30.352939 4824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:30.353232 4824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:30.356108 4824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "75d0e8f196ec492088afc01525fe96ad"
format_stamp: "Formatted at 2026-08-07 04:22:30 on dist-test-slave-zkpd"
server_key: "e1657989bf107065fd5895b0acd2aad4"
server_key_iv: "5ec6b3a6a1efc0047b167cb799023fd3"
server_key_version: "encryptionkey@0"
I20260807 04:22:30.356910 4824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "75d0e8f196ec492088afc01525fe96ad"
format_stamp: "Formatted at 2026-08-07 04:22:30 on dist-test-slave-zkpd"
server_key: "e1657989bf107065fd5895b0acd2aad4"
server_key_iv: "5ec6b3a6a1efc0047b167cb799023fd3"
server_key_version: "encryptionkey@0"
I20260807 04:22:30.362051 4824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:30.365275 4839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:30.366488 4824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 04:22:30.366660 4824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "75d0e8f196ec492088afc01525fe96ad"
format_stamp: "Formatted at 2026-08-07 04:22:30 on dist-test-slave-zkpd"
server_key: "e1657989bf107065fd5895b0acd2aad4"
server_key_iv: "5ec6b3a6a1efc0047b167cb799023fd3"
server_key_version: "encryptionkey@0"
I20260807 04:22:30.366852 4824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:30.401840 4824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:30.405861 4824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:30.406155 4824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:30.407143 4824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:30.408602 4824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:30.408705 4824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:30.408869 4824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:30.408942 4824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:30.422500 4680 heartbeater.cc:503] Master 127.1.244.254:45505 was elected leader, sending a full tablet report...
I20260807 04:22:30.442859 4824 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39999
I20260807 04:22:30.442907 4955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39999 every 8 connection(s)
I20260807 04:22:30.444255 4824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:30.449383 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 4824
I20260807 04:22:30.449601 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:30.450132 2003 external_mini_cluster.cc:1428] Setting key cb4f53a3953a5a4fd772bf9a86f880fe
Aug 07 04:22:30 dist-test-slave-zkpd krb5kdc[4440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076550, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:30.464339 4820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:30.446667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:48821 (local address 127.1.244.254:45505)
0807 04:22:30.446862 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:30.446867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:30.447980 (+ 1113us) server_negotiation.cc:407] Connection header received
0807 04:22:30.450320 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:30.450332 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:30.450412 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:30.450492 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:30.452177 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:30.453054 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:30.454589 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:30.454973 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:30.455068 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:30.458603 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:30.458608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:30.458612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:30.458644 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:30.461360 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:30.462136 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:30.462140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:30.462142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:30.462261 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:30.462631 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:30.462634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:30.462636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:30.462886 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:30.463004 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:30.463674 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 04:22:30.463885 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:22:30.465508 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:30.447078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45505 (local address 127.1.244.195:48821)
0807 04:22:30.447806 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:30.447843 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:30.449997 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:30.450778 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:30.450787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:30.451274 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:30.452036 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:30.452056 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:30.453208 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:30.453213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:30.454464 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:30.454476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:30.455156 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:30.455161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:30.455357 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:30.455996 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:30.456078 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:30.458388 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:30.461579 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:30.461586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:30.461597 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:30.462022 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:30.462351 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:30.462355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:30.462357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:30.462521 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:30.462987 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:30.462992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:30.463274 (+ 282us) client_negotiation.cc:769] Sending connection context
0807 04:22:30.463514 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 04:22:30.464077 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":274,"threads_started":1}
I20260807 04:22:30.467068 4956 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45505
I20260807 04:22:30.467397 4956 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:30.468160 4956 heartbeater.cc:511] Master 127.1.244.254:45505 requested a full tablet report, sending...
I20260807 04:22:30.469475 4487 ts_manager.cc:194] Registered new tserver with Master: 75d0e8f196ec492088afc01525fe96ad (127.1.244.195:39999)
I20260807 04:22:30.470366 4487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:48821
I20260807 04:22:30.478021 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 04:22:30.510259 4820 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:30.508049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39120 (local address 127.1.244.254:45505)
0807 04:22:30.508219 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:30.508223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:30.508649 (+ 426us) server_negotiation.cc:407] Connection header received
0807 04:22:30.508792 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:30.508795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:30.508870 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:30.508968 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:30.510059 (+ 1091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39120: BlockingRecv error: recv got EOF from 127.0.0.1:39120 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:22:30.513865 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4549
W20260807 04:22:30.516513 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4549: Not authorized: Client connection negotiation failed: client connection to 127.1.244.193:42857: server requires authentication, but client does not have Kerberos credentials available
I20260807 04:22:30.516685 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 4549
I20260807 04:22:30.535347 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4686
W20260807 04:22:30.538002 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4686: Not authorized: Client connection negotiation failed: client connection to 127.1.244.194:37919: server requires authentication, but client does not have Kerberos credentials available
I20260807 04:22:30.538168 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 4686
I20260807 04:22:30.554971 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4824
W20260807 04:22:30.558055 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4824: Not authorized: Client connection negotiation failed: client connection to 127.1.244.195:39999: server requires authentication, but client does not have Kerberos credentials available
I20260807 04:22:30.558193 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 4824
I20260807 04:22:30.575286 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4454
I20260807 04:22:30.576524 4517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:30.678253 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 4454
2026-08-07T04:22:30Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3607 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:30 dist-test-slave-zkpd krb5kdc[4973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:30 dist-test-slave-zkpd krb5kdc[4973](info): set up 2 sockets
Aug 07 04:22:30 dist-test-slave-zkpd krb5kdc[4973](info): commencing operation
krb5kdc: starting...
W20260807 04:22:31.786577 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:31 dist-test-slave-zkpd krb5kdc[4973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:31.987017 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39633
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:32.306516 4986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:32.306818 4986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:32.306972 4986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:32.313956 4986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:32.314096 4986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:32.314180 4986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:32.314265 4986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:32.314338 4986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:32.323947 4986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:32.325248 4986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:32.326543 4986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:32.338302 4991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:32.338354 4992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:32.338375 4994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:32.339221 4986 server_base.cc:1034] running on GCE node
I20260807 04:22:32.340072 4986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:32.341413 4986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:32.342644 4986 hybrid_clock.cc:648] HybridClock initialized: now 1786076552342593 us; error 38 us; skew 500 ppm
I20260807 04:22:32.342999 4986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:32 dist-test-slave-zkpd krb5kdc[4973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076552, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:32.346172 4986 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:32.347539 4986 webserver.cc:460] Webserver started at http://127.1.244.254:37657/ using document root <none> and password file <none>
I20260807 04:22:32.348285 4986 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:32.348438 4986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:32.348753 4986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:32.351629 4986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "1156c8df941c442fa73efd42f946ad6c"
format_stamp: "Formatted at 2026-08-07 04:22:32 on dist-test-slave-zkpd"
server_key: "9532aa1168702f74f0a72b9a0fcd0c9e"
server_key_iv: "42b6b71dbdef2aa8b9bc4b1244961b93"
server_key_version: "encryptionkey@0"
I20260807 04:22:32.352481 4986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1156c8df941c442fa73efd42f946ad6c"
format_stamp: "Formatted at 2026-08-07 04:22:32 on dist-test-slave-zkpd"
server_key: "9532aa1168702f74f0a72b9a0fcd0c9e"
server_key_iv: "42b6b71dbdef2aa8b9bc4b1244961b93"
server_key_version: "encryptionkey@0"
I20260807 04:22:32.357349 4986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:22:32.360594 5002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:32.361824 4986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:22:32.362002 4986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "1156c8df941c442fa73efd42f946ad6c"
format_stamp: "Formatted at 2026-08-07 04:22:32 on dist-test-slave-zkpd"
server_key: "9532aa1168702f74f0a72b9a0fcd0c9e"
server_key_iv: "42b6b71dbdef2aa8b9bc4b1244961b93"
server_key_version: "encryptionkey@0"
I20260807 04:22:32.362192 4986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:32.375715 4986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:32.380108 4986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:32.380389 4986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:32.398108 4986 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39633
I20260807 04:22:32.398144 5057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39633 every 8 connection(s)
I20260807 04:22:32.399535 4986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:32.403184 5058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:32.405328 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 4986
I20260807 04:22:32.405539 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:32.406164 2003 external_mini_cluster.cc:1428] Setting key bf18803b425a055eda8d01b025e726b4
I20260807 04:22:32.415318 5058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c: Bootstrap starting.
I20260807 04:22:32.419114 5058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:32.420468 5058 log.cc:824] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:22:32 dist-test-slave-zkpd krb5kdc[4973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:32.423252 5058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c: No bootstrap required, opened a new log
I20260807 04:22:32.430088 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:32.408059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.1.244.254:39633)
0807 04:22:32.408904 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:32.408915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:32.408956 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:22:32.410605 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:32.410629 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:32.411011 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:32.411354 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:32.413378 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:32.414656 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:32.416376 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:32.417062 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:32.417200 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:32.422245 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:32.422253 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:32.422266 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:32.422303 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:32.425610 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:32.426356 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:32.426366 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:32.426374 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:32.426625 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:32.427059 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:32.427062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:32.427064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:32.427640 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:32.427865 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:32.428532 (+ 667us) server_negotiation.cc:299] Negotiation successful
0807 04:22:32.428993 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":422,"threads_started":1}
I20260807 04:22:32.432144 5058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1156c8df941c442fa73efd42f946ad6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39633 } }
I20260807 04:22:32.432507 5058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:32.432631 5058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1156c8df941c442fa73efd42f946ad6c, State: Initialized, Role: FOLLOWER
I20260807 04:22:32.433306 5058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1156c8df941c442fa73efd42f946ad6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39633 } }
I20260807 04:22:32.433599 5058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:32.433763 5058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:32.433954 5058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:32.434342 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:22:32.436214 5058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1156c8df941c442fa73efd42f946ad6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39633 } }
I20260807 04:22:32.436690 5058 leader_election.cc:302] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1156c8df941c442fa73efd42f946ad6c; no voters:
I20260807 04:22:32.437290 5058 leader_election.cc:288] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:32.437438 5063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:32.439206 5058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:32.444291 5063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [term 1 LEADER]: Becoming Leader. State: Replica: 1156c8df941c442fa73efd42f946ad6c, State: Running, Role: LEADER
I20260807 04:22:32.444798 5063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1156c8df941c442fa73efd42f946ad6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39633 } }
I20260807 04:22:32.447364 5065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1156c8df941c442fa73efd42f946ad6c. Latest consensus state: current_term: 1 leader_uuid: "1156c8df941c442fa73efd42f946ad6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1156c8df941c442fa73efd42f946ad6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39633 } } }
I20260807 04:22:32.447397 5064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1156c8df941c442fa73efd42f946ad6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1156c8df941c442fa73efd42f946ad6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39633 } } }
I20260807 04:22:32.447822 5065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:32.447913 5064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:32.449448 5078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:32.458703 5078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:32.467286 5078 catalog_manager.cc:1347] Generated new cluster ID: e9810c0332154a62a284c201c2a89dbb
I20260807 04:22:32.467411 5078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:32.485524 5078 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:32.486910 5061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:32.477825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.1.244.254:39633)
0807 04:22:32.478008 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:32.478016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:32.478599 (+ 583us) server_negotiation.cc:407] Connection header received
0807 04:22:32.478808 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:32.478819 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:32.478919 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:32.479039 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:32.480386 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:32.481487 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:32.483441 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:32.484076 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:32.484209 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:32.485320 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:32.485327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:32.485332 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:32.486069 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:32.486130 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:32.486162 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:32.486287 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:32.486460 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 04:22:32.486555 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:22:32.487521 5078 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:32.499707 5078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1156c8df941c442fa73efd42f946ad6c: Generated new TSK 0
I20260807 04:22:32.500694 5078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:22:32.510857 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 4986
I20260807 04:22:32.512084 5053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:32.600088 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 4986
2026-08-07T04:22:32Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1916 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:32 dist-test-slave-zkpd krb5kdc[5094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:32 dist-test-slave-zkpd krb5kdc[5094](info): set up 2 sockets
Aug 07 04:22:32 dist-test-slave-zkpd krb5kdc[5094](info): commencing operation
krb5kdc: starting...
W20260807 04:22:33.699590 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:33 dist-test-slave-zkpd krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:33.906689 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45569
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:34.231258 5107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:34.231566 5107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:34.231750 5107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:34.238878 5107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:34.239023 5107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:34.239131 5107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:34.239240 5107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:34.239311 5107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:34.248896 5107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:34.250195 5107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:34.251471 5107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:34.263307 5115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:34.263522 5113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:34.263940 5112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:34.264474 5107 server_base.cc:1034] running on GCE node
I20260807 04:22:34.265272 5107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:34.266629 5107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:34.267937 5107 hybrid_clock.cc:648] HybridClock initialized: now 1786076554267838 us; error 57 us; skew 500 ppm
I20260807 04:22:34.268299 5107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:34 dist-test-slave-zkpd krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:34.275878 5107 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:34.277371 5107 webserver.cc:460] Webserver started at http://127.1.244.254:39373/ using document root <none> and password file <none>
I20260807 04:22:34.278047 5107 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:34.278204 5107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:34.278527 5107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:34.281605 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "f549eb73738d462bbe8cc221b6d5d152"
format_stamp: "Formatted at 2026-08-07 04:22:34 on dist-test-slave-zkpd"
server_key: "ac6c8359890c22fdfb456e875f135663"
server_key_iv: "2b2614d4b4fa09040b21695752876c6a"
server_key_version: "encryptionkey@0"
I20260807 04:22:34.282436 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "f549eb73738d462bbe8cc221b6d5d152"
format_stamp: "Formatted at 2026-08-07 04:22:34 on dist-test-slave-zkpd"
server_key: "ac6c8359890c22fdfb456e875f135663"
server_key_iv: "2b2614d4b4fa09040b21695752876c6a"
server_key_version: "encryptionkey@0"
I20260807 04:22:34.287745 5107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:22:34.291038 5122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:34.292304 5107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:34.292482 5107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "f549eb73738d462bbe8cc221b6d5d152"
format_stamp: "Formatted at 2026-08-07 04:22:34 on dist-test-slave-zkpd"
server_key: "ac6c8359890c22fdfb456e875f135663"
server_key_iv: "2b2614d4b4fa09040b21695752876c6a"
server_key_version: "encryptionkey@0"
I20260807 04:22:34.292671 5107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:34.325103 5107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:34.329681 5107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:34.329972 5107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:34.348338 5107 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45569
I20260807 04:22:34.348379 5175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45569 every 8 connection(s)
I20260807 04:22:34.349675 5107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:34.353197 5176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:34.355352 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5107
I20260807 04:22:34.355568 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:34.356281 2003 external_mini_cluster.cc:1428] Setting key 8646a973a32608d7d16f44ad75397c49
I20260807 04:22:34.365759 5176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152: Bootstrap starting.
I20260807 04:22:34.369051 5176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:34.370009 5176 log.cc:824] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:22:34 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:34.372872 5176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152: No bootstrap required, opened a new log
I20260807 04:22:34.380831 5180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:34.358291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.1.244.254:45569)
0807 04:22:34.359085 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:34.359100 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:34.359146 (+ 46us) server_negotiation.cc:407] Connection header received
0807 04:22:34.361532 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:34.361563 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:34.362044 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:34.362477 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:34.363769 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:34.365699 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:34.366983 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:34.368141 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:34.368346 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:34.372389 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:34.372401 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:34.372415 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:34.372448 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:34.376460 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:34.377112 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:34.377121 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:34.377130 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:34.377348 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:34.377710 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:34.377715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:34.377719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:34.378384 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:34.378730 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:34.379007 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 04:22:34.379452 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":306,"threads_started":1}
I20260807 04:22:34.383023 5176 raft_consensus.cc:348] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f549eb73738d462bbe8cc221b6d5d152" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45569 } }
I20260807 04:22:34.383384 5176 raft_consensus.cc:374] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:34.383495 5176 raft_consensus.cc:729] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f549eb73738d462bbe8cc221b6d5d152, State: Initialized, Role: FOLLOWER
I20260807 04:22:34.384160 5176 consensus_queue.cc:260] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f549eb73738d462bbe8cc221b6d5d152" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45569 } }
I20260807 04:22:34.384415 5176 raft_consensus.cc:388] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:34.384559 5176 raft_consensus.cc:482] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:34.384732 5176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:34.387020 5176 raft_consensus.cc:504] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f549eb73738d462bbe8cc221b6d5d152" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45569 } }
I20260807 04:22:34.387492 5176 leader_election.cc:302] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f549eb73738d462bbe8cc221b6d5d152; no voters:
I20260807 04:22:34.388098 5176 leader_election.cc:288] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:34.388190 5182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:34.388939 5182 raft_consensus.cc:686] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [term 1 LEADER]: Becoming Leader. State: Replica: f549eb73738d462bbe8cc221b6d5d152, State: Running, Role: LEADER
I20260807 04:22:34.389489 5182 consensus_queue.cc:237] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f549eb73738d462bbe8cc221b6d5d152" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45569 } }
I20260807 04:22:34.390156 5176 sys_catalog.cc:564] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:34.392223 5183 sys_catalog.cc:455] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f549eb73738d462bbe8cc221b6d5d152" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f549eb73738d462bbe8cc221b6d5d152" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45569 } } }
I20260807 04:22:34.392627 5183 sys_catalog.cc:458] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:34.396886 5184 sys_catalog.cc:455] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f549eb73738d462bbe8cc221b6d5d152. Latest consensus state: current_term: 1 leader_uuid: "f549eb73738d462bbe8cc221b6d5d152" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f549eb73738d462bbe8cc221b6d5d152" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45569 } } }
I20260807 04:22:34.397328 5184 sys_catalog.cc:458] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:34.398527 5188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:34.402648 5188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:34.410591 5188 catalog_manager.cc:1347] Generated new cluster ID: 4b0f9f3dcf7e4ffbaac0c60893bd122d
I20260807 04:22:34.410718 5188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:34.422077 5188 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:34.423841 5188 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:34.441784 5188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f549eb73738d462bbe8cc221b6d5d152: Generated new TSK 0
I20260807 04:22:34.442586 5188 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:34.501482 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45569
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:34.822611 5204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:34.822897 5204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:34.823060 5204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:34.830188 5204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:34.830322 5204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:34.830502 5204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:34.840528 5204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:34.841755 5204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:34.843021 5204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:34.856074 5209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:34.856038 5210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:34.856240 5212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:34.857043 5204 server_base.cc:1034] running on GCE node
I20260807 04:22:34.857685 5204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:34.858667 5204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:34.859941 5204 hybrid_clock.cc:648] HybridClock initialized: now 1786076554859831 us; error 45 us; skew 500 ppm
I20260807 04:22:34.860249 5204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:34 dist-test-slave-zkpd krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:34.863188 5204 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:34.864591 5204 webserver.cc:460] Webserver started at http://127.1.244.193:35021/ using document root <none> and password file <none>
I20260807 04:22:34.865166 5204 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:34.865269 5204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:34.865536 5204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:34.868296 5204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "6df7f32f9d084758b7b035437812d818"
format_stamp: "Formatted at 2026-08-07 04:22:34 on dist-test-slave-zkpd"
server_key: "c3779d86ebc6c78fd498926c23fb81e9"
server_key_iv: "400a159f00e819d8fda4bd14b7abaea0"
server_key_version: "encryptionkey@0"
I20260807 04:22:34.869005 5204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "6df7f32f9d084758b7b035437812d818"
format_stamp: "Formatted at 2026-08-07 04:22:34 on dist-test-slave-zkpd"
server_key: "c3779d86ebc6c78fd498926c23fb81e9"
server_key_iv: "400a159f00e819d8fda4bd14b7abaea0"
server_key_version: "encryptionkey@0"
I20260807 04:22:34.873746 5204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:22:34.876927 5219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:34.878153 5204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 04:22:34.878286 5204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "6df7f32f9d084758b7b035437812d818"
format_stamp: "Formatted at 2026-08-07 04:22:34 on dist-test-slave-zkpd"
server_key: "c3779d86ebc6c78fd498926c23fb81e9"
server_key_iv: "400a159f00e819d8fda4bd14b7abaea0"
server_key_version: "encryptionkey@0"
I20260807 04:22:34.878436 5204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:34.916589 5204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:34.920260 5204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:34.920521 5204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:34.921428 5204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:34.922684 5204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:34.922755 5204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:34.922833 5204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:34.922883 5204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:34.954319 5204 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:43087
I20260807 04:22:34.954365 5340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:43087 every 8 connection(s)
I20260807 04:22:34.955588 5204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:34.962378 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5204
I20260807 04:22:34.962579 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:34.963434 2003 external_mini_cluster.cc:1428] Setting key e95db7acc1eceda5feb2b84609d1abc3
Aug 07 04:22:34 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076554, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:34.978514 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:34.958124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:40103 (local address 127.1.244.254:45569)
0807 04:22:34.958564 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:34.958569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:34.959530 (+ 961us) server_negotiation.cc:407] Connection header received
0807 04:22:34.961288 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:34.961292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:34.961377 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:34.961480 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:34.963033 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:34.964163 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:34.966127 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:34.966856 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:34.967008 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:34.970359 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:34.970368 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:34.970375 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:34.970428 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:34.974233 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:34.975327 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:34.975333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:34.975336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:34.975545 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:34.976168 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:34.976173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:34.976176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:34.976617 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:34.976770 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:34.977858 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0807 04:22:34.978126 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":231,"threads_started":1}
I20260807 04:22:34.979664 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:34.958704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45569 (local address 127.1.244.193:40103)
0807 04:22:34.959374 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:34.959418 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:34.961056 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:34.961668 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:34.961676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:34.962150 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:34.962834 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:34.962853 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:34.964341 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:34.964345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:34.965975 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:34.965987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:34.967075 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:34.967079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:34.967274 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:34.968018 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:34.968041 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:34.970147 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:34.974414 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:34.974425 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:34.974440 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:34.975182 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:34.975722 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:34.975727 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:34.975731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:34.976046 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:34.976799 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:34.976810 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:34.977342 (+ 532us) client_negotiation.cc:769] Sending connection context
0807 04:22:34.977704 (+ 362us) client_negotiation.cc:240] Negotiation successful
0807 04:22:34.978246 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":266,"threads_started":1}
I20260807 04:22:34.984854 5341 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45569
I20260807 04:22:34.985193 5341 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:34.985977 5341 heartbeater.cc:511] Master 127.1.244.254:45569 requested a full tablet report, sending...
I20260807 04:22:34.988195 5140 ts_manager.cc:194] Registered new tserver with Master: 6df7f32f9d084758b7b035437812d818 (127.1.244.193:43087)
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260807 04:22:34.989879 5140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:40103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:35.017038 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45569
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:35.343840 5347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:35.344198 5347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:35.344368 5347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:35.351639 5347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:35.351799 5347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:35.352017 5347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:35.361626 5347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:35.362807 5347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:35.364156 5347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:35.377501 5352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:35.377463 5353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:35.377542 5355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:35.378300 5347 server_base.cc:1034] running on GCE node
I20260807 04:22:35.378880 5347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:35.379742 5347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:35.380961 5347 hybrid_clock.cc:648] HybridClock initialized: now 1786076555380907 us; error 67 us; skew 500 ppm
I20260807 04:22:35.381322 5347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:35 dist-test-slave-zkpd krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:35.384428 5347 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:35.385677 5347 webserver.cc:460] Webserver started at http://127.1.244.194:33043/ using document root <none> and password file <none>
I20260807 04:22:35.386281 5347 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:35.386380 5347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:35.386622 5347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:35.389428 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "ae024e6527aa496e870200cd9f08f225"
format_stamp: "Formatted at 2026-08-07 04:22:35 on dist-test-slave-zkpd"
server_key: "eff8afab2508554dcce5f1326e60dfcb"
server_key_iv: "4b132c843dff4238fa3bc66f2ddde86e"
server_key_version: "encryptionkey@0"
I20260807 04:22:35.390202 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "ae024e6527aa496e870200cd9f08f225"
format_stamp: "Formatted at 2026-08-07 04:22:35 on dist-test-slave-zkpd"
server_key: "eff8afab2508554dcce5f1326e60dfcb"
server_key_iv: "4b132c843dff4238fa3bc66f2ddde86e"
server_key_version: "encryptionkey@0"
I20260807 04:22:35.395376 5347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:22:35.398706 5362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:35.400110 5347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:35.400262 5347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "ae024e6527aa496e870200cd9f08f225"
format_stamp: "Formatted at 2026-08-07 04:22:35 on dist-test-slave-zkpd"
server_key: "eff8afab2508554dcce5f1326e60dfcb"
server_key_iv: "4b132c843dff4238fa3bc66f2ddde86e"
server_key_version: "encryptionkey@0"
I20260807 04:22:35.400456 5347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:35.419452 5347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:35.423200 5347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:35.423516 5347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:35.424515 5347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:35.425807 5347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:35.425879 5347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:35.426049 5347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:35.426116 5347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:35.457959 5347 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:46125
I20260807 04:22:35.458060 5479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:46125 every 8 connection(s)
I20260807 04:22:35.459678 5347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:35.468447 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5347
I20260807 04:22:35.468636 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:35.469075 2003 external_mini_cluster.cc:1428] Setting key c5d285810f227f67e6cfdb18444af5e1
Aug 07 04:22:35 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:35.481086 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:35.462573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:57255 (local address 127.1.244.254:45569)
0807 04:22:35.462818 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:35.462825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:35.464819 (+ 1994us) server_negotiation.cc:407] Connection header received
0807 04:22:35.466871 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:35.466875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:35.466949 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:35.467040 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:35.468502 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.469374 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.471482 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.471977 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.472111 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:35.475477 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:35.475483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:35.475486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:35.475517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:35.478115 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:35.478811 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:35.478817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:35.478821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:35.478963 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:35.479375 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:35.479381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:35.479384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:35.479632 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:35.479781 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:35.480485 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 04:22:35.480733 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:22:35.482133 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:35.463214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45569 (local address 127.1.244.194:57255)
0807 04:22:35.464637 (+ 1423us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:35.464696 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:35.466622 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:35.467240 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:35.467247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:35.467724 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:35.468326 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:35.468344 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.469525 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.469531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:35.471306 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:35.471318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.472199 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.472206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:35.472437 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:35.473114 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:35.473140 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:35.475260 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:35.478254 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:35.478262 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:35.478277 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:35.478689 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:35.479074 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:35.479080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:35.479084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:35.479265 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:35.479768 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:35.479776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:35.480142 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 04:22:35.480410 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 04:22:35.480912 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":825,"threads_started":1}
I20260807 04:22:35.483577 5480 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45569
I20260807 04:22:35.483981 5480 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:35.484861 5480 heartbeater.cc:511] Master 127.1.244.254:45569 requested a full tablet report, sending...
I20260807 04:22:35.486418 5140 ts_manager.cc:194] Registered new tserver with Master: ae024e6527aa496e870200cd9f08f225 (127.1.244.194:46125)
I20260807 04:22:35.487202 5140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:57255
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:35.515165 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45569
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:35.837792 5486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:35.838053 5486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:35.838143 5486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:35.845297 5486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:35.845404 5486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:35.845516 5486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:35.855140 5486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45569
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:35.856323 5486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:35.857497 5486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:35.870940 5492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:35.871158 5491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:35.871174 5495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:35.871999 5486 server_base.cc:1034] running on GCE node
I20260807 04:22:35.872622 5486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:35.873490 5486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:35.874713 5486 hybrid_clock.cc:648] HybridClock initialized: now 1786076555874651 us; error 65 us; skew 500 ppm
I20260807 04:22:35.875025 5486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:35 dist-test-slave-zkpd krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:35.878012 5486 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:35.879454 5486 webserver.cc:460] Webserver started at http://127.1.244.195:33447/ using document root <none> and password file <none>
I20260807 04:22:35.880172 5486 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:35.880340 5486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:35.880633 5486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:35.883540 5486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "2f61f10b2884498c9a82de3b563d564f"
format_stamp: "Formatted at 2026-08-07 04:22:35 on dist-test-slave-zkpd"
server_key: "3eb13ba9070313a7ba503ee37db2b17a"
server_key_iv: "4763072fd01ab601296a0c5c1bb5f217"
server_key_version: "encryptionkey@0"
I20260807 04:22:35.884310 5486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "2f61f10b2884498c9a82de3b563d564f"
format_stamp: "Formatted at 2026-08-07 04:22:35 on dist-test-slave-zkpd"
server_key: "3eb13ba9070313a7ba503ee37db2b17a"
server_key_iv: "4763072fd01ab601296a0c5c1bb5f217"
server_key_version: "encryptionkey@0"
I20260807 04:22:35.889045 5486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:22:35.892396 5502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:35.893637 5486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:22:35.893839 5486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "2f61f10b2884498c9a82de3b563d564f"
format_stamp: "Formatted at 2026-08-07 04:22:35 on dist-test-slave-zkpd"
server_key: "3eb13ba9070313a7ba503ee37db2b17a"
server_key_iv: "4763072fd01ab601296a0c5c1bb5f217"
server_key_version: "encryptionkey@0"
I20260807 04:22:35.894037 5486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:35.912425 5486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:35.916185 5486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:35.916476 5486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:35.917377 5486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:35.918643 5486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:35.918745 5486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:35.918916 5486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:35.918994 5486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:35.951807 5486 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:43017
I20260807 04:22:35.951846 5619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:43017 every 8 connection(s)
I20260807 04:22:35.953181 5486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:35.963385 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5486
I20260807 04:22:35.963590 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:35.964128 2003 external_mini_cluster.cc:1428] Setting key 149b11832d29398d907a14c957989b50
Aug 07 04:22:35 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076555, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:35.973927 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:35.956073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:50101 (local address 127.1.244.254:45569)
0807 04:22:35.956294 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:35.956301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:35.957511 (+ 1210us) server_negotiation.cc:407] Connection header received
0807 04:22:35.960006 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:35.960010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:35.960093 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:35.960201 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:35.961756 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.962506 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.964166 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.964770 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.964890 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:35.968562 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:35.968566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:35.968569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:35.968600 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:35.970960 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:35.971605 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:35.971608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:35.971611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:35.971751 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:35.972236 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:35.972242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:35.972250 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:35.972525 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:35.972682 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:35.973346 (+ 664us) server_negotiation.cc:299] Negotiation successful
0807 04:22:35.973555 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:22:35.974802 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:35.956529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45569 (local address 127.1.244.195:50101)
0807 04:22:35.957325 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:35.957368 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:35.959697 (+ 2329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:35.960453 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:35.960460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:35.960937 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:35.961453 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:35.961470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.962637 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.962641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:35.964021 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:35.964033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:35.965053 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:35.965061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:35.965323 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:35.966073 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:35.966111 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:35.968373 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:35.971079 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:35.971085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:35.971096 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:35.971505 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:35.971873 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:35.971878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:35.971882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:35.972113 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:35.972659 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:35.972667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:35.972984 (+ 317us) client_negotiation.cc:769] Sending connection context
0807 04:22:35.973255 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 04:22:35.973774 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":312,"threads_started":1}
I20260807 04:22:35.976125 5620 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45569
I20260807 04:22:35.976411 5620 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:35.977111 5620 heartbeater.cc:511] Master 127.1.244.254:45569 requested a full tablet report, sending...
I20260807 04:22:35.978643 5140 ts_manager.cc:194] Registered new tserver with Master: 2f61f10b2884498c9a82de3b563d564f (127.1.244.195:43017)
I20260807 04:22:35.979436 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:22:35.979615 5140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:50101
I20260807 04:22:35.993536 5341 heartbeater.cc:503] Master 127.1.244.254:45569 was elected leader, sending a full tablet report...
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:36.031590 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:36.017256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.1.244.254:45569)
0807 04:22:36.017497 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:36.017505 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:36.017937 (+ 432us) server_negotiation.cc:407] Connection header received
0807 04:22:36.018155 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:36.018159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:36.018225 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:36.018332 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:36.019757 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.020605 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.021873 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.022311 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.022451 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:36.026316 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:36.026321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:36.026325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:36.026358 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:36.028959 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.029604 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.029616 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.029619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.029771 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.030157 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.030162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.030167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.030488 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:36.030759 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:36.031038 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 04:22:36.031249 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 04:22:36.032191 5140 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35736
I20260807 04:22:36.034066 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5204
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:36.051239 5631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:36.035251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.1.244.193:43087)
0807 04:22:36.038007 (+ 2756us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:36.038014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:36.038038 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:22:36.038092 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:36.038099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:36.038307 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:36.038428 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:36.039557 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.040602 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.041716 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.042244 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.042335 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:36.046256 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:36.046266 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:36.046278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:36.046307 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:36.048921 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.049509 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.049516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.049518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.049631 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.049933 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.049936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.049938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.050231 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:36.050445 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:36.050670 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 04:22:36.050850 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2611,"thread_start_us":2462,"threads_started":1}
W20260807 04:22:36.052737 5335 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57520
W20260807 04:22:36.054085 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5204: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:36.054320 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5204
I20260807 04:22:36.072562 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5347
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:36.088585 5633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:36.073340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.1.244.194:46125)
0807 04:22:36.073886 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:36.073893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:36.073919 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:22:36.073980 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:36.073987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:36.074195 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:36.074326 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:36.075571 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.076625 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.077834 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.078340 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.078442 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:36.082459 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:36.082471 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:36.082484 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:36.082514 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:36.085772 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.086700 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.086706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.086709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.086832 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.087129 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.087132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.087134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.087479 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:36.087704 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:36.087995 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 04:22:36.088189 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":275,"threads_started":1}
W20260807 04:22:36.089902 5474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38180
W20260807 04:22:36.090921 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5347: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:36.091059 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5347
I20260807 04:22:36.111455 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5486
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076556, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:36.127321 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:36.112465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.1.244.195:43017)
0807 04:22:36.113131 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:36.113137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:36.113165 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:22:36.113226 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:36.113232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:36.113432 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:36.113601 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:36.114941 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.116061 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.117271 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:36.117808 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:36.117950 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:36.122154 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:36.122165 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:36.122178 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:36.122214 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:36.125107 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.125673 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.125679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.125681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.125799 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:36.126090 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:36.126093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:36.126095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:36.126428 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:36.126635 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:36.126850 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 04:22:36.127039 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":365,"threads_started":1}
W20260807 04:22:36.128679 5615 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36634
W20260807 04:22:36.129699 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5486: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:22:36.129866 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5486
I20260807 04:22:36.147711 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5107
I20260807 04:22:36.148929 5171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:36.254304 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5107
2026-08-07T04:22:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3661 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5638](info): set up 2 sockets
Aug 07 04:22:36 dist-test-slave-zkpd krb5kdc[5638](info): commencing operation
krb5kdc: starting...
W20260807 04:22:37.374161 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:37 dist-test-slave-zkpd krb5kdc[5638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:37.576550 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46219
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:37.900218 5653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:37.900492 5653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:37.900708 5653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:37.907951 5653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:37.908080 5653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:37.908169 5653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:37.908267 5653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:37.908363 5653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:37.917932 5653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:37.919170 5653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:37.920547 5653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:37.932330 5658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:37.932435 5660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:37.932396 5662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:37.933209 5653 server_base.cc:1034] running on GCE node
I20260807 04:22:37.933992 5653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:37.935282 5653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:37.936508 5653 hybrid_clock.cc:648] HybridClock initialized: now 1786076557936482 us; error 44 us; skew 500 ppm
I20260807 04:22:37.936853 5653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:37 dist-test-slave-zkpd krb5kdc[5638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076557, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:37.940099 5653 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:37.941540 5653 webserver.cc:460] Webserver started at http://127.1.244.254:42841/ using document root <none> and password file <none>
I20260807 04:22:37.942194 5653 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:37.942349 5653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:37.942698 5653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:37.945716 5653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "d013f1ef66584ed380ba18c5916f6d6c"
format_stamp: "Formatted at 2026-08-07 04:22:37 on dist-test-slave-zkpd"
server_key: "c489cb90c6bf2ec655a6e49da08e6810"
server_key_iv: "a00a0a7eaaad38fb99e081480c06fd62"
server_key_version: "encryptionkey@0"
I20260807 04:22:37.946516 5653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "d013f1ef66584ed380ba18c5916f6d6c"
format_stamp: "Formatted at 2026-08-07 04:22:37 on dist-test-slave-zkpd"
server_key: "c489cb90c6bf2ec655a6e49da08e6810"
server_key_iv: "a00a0a7eaaad38fb99e081480c06fd62"
server_key_version: "encryptionkey@0"
I20260807 04:22:37.951601 5653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 04:22:37.954929 5670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:37.956218 5653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:37.956393 5653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "d013f1ef66584ed380ba18c5916f6d6c"
format_stamp: "Formatted at 2026-08-07 04:22:37 on dist-test-slave-zkpd"
server_key: "c489cb90c6bf2ec655a6e49da08e6810"
server_key_iv: "a00a0a7eaaad38fb99e081480c06fd62"
server_key_version: "encryptionkey@0"
I20260807 04:22:37.956596 5653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:37.975672 5653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:37.979961 5653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:37.980288 5653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:37.998798 5653 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46219
I20260807 04:22:37.998837 5723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46219 every 8 connection(s)
I20260807 04:22:38.000258 5653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:38.003801 5724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:38.005486 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5653
I20260807 04:22:38.005751 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:38.006563 2003 external_mini_cluster.cc:1428] Setting key eea3e1baec9504ec7f8cceb78aa4423a
I20260807 04:22:38.016706 5724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c: Bootstrap starting.
I20260807 04:22:38.019526 5724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c: Neither blocks nor log segments found. Creating new log.
Aug 07 04:22:38 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:38.020646 5724 log.cc:824] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:38.023329 5724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c: No bootstrap required, opened a new log
I20260807 04:22:38.028573 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:38.008697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.1.244.254:46219)
0807 04:22:38.009505 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:38.009516 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:38.009556 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:22:38.011158 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:38.011182 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:38.011548 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:38.011960 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:38.013404 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:38.014754 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:38.016171 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:38.016848 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:38.017067 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:38.021148 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:38.021176 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:38.021189 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:38.021227 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:38.024475 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:38.025148 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:38.025155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:38.025161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:38.025332 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:38.025776 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:38.025780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:38.025782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:38.026299 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:38.026651 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:38.027145 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 04:22:38.027549 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":393,"threads_started":1}
I20260807 04:22:38.031457 5724 raft_consensus.cc:348] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d013f1ef66584ed380ba18c5916f6d6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46219 } }
I20260807 04:22:38.031817 5724 raft_consensus.cc:374] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:38.031936 5724 raft_consensus.cc:729] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d013f1ef66584ed380ba18c5916f6d6c, State: Initialized, Role: FOLLOWER
I20260807 04:22:38.032508 5724 consensus_queue.cc:260] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d013f1ef66584ed380ba18c5916f6d6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46219 } }
I20260807 04:22:38.032783 5724 raft_consensus.cc:388] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:38.032923 5724 raft_consensus.cc:482] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:38.033103 5724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:38.035597 5724 raft_consensus.cc:504] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d013f1ef66584ed380ba18c5916f6d6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46219 } }
I20260807 04:22:38.036175 5724 leader_election.cc:302] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d013f1ef66584ed380ba18c5916f6d6c; no voters:
I20260807 04:22:38.036785 5724 leader_election.cc:288] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:38.036878 5729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:38.037724 5729 raft_consensus.cc:686] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [term 1 LEADER]: Becoming Leader. State: Replica: d013f1ef66584ed380ba18c5916f6d6c, State: Running, Role: LEADER
I20260807 04:22:38.038285 5729 consensus_queue.cc:237] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d013f1ef66584ed380ba18c5916f6d6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46219 } }
I20260807 04:22:38.039098 5724 sys_catalog.cc:564] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:38.041246 5730 sys_catalog.cc:455] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d013f1ef66584ed380ba18c5916f6d6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d013f1ef66584ed380ba18c5916f6d6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46219 } } }
I20260807 04:22:38.041705 5730 sys_catalog.cc:458] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:38.041247 5731 sys_catalog.cc:455] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d013f1ef66584ed380ba18c5916f6d6c. Latest consensus state: current_term: 1 leader_uuid: "d013f1ef66584ed380ba18c5916f6d6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d013f1ef66584ed380ba18c5916f6d6c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46219 } } }
I20260807 04:22:38.043491 5731 sys_catalog.cc:458] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:38.046530 5734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:38.051772 5734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:38.060444 5734 catalog_manager.cc:1347] Generated new cluster ID: dd20b7ab3f554ad59d6cce2013ec7af7
I20260807 04:22:38.060571 5734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:38.097579 5734 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:38.098984 5734 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:38.109781 5734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d013f1ef66584ed380ba18c5916f6d6c: Generated new TSK 0
I20260807 04:22:38.110607 5734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:38.156878 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46219
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:38.480175 5751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:38.480427 5751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:38.480594 5751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:38.487679 5751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:38.487802 5751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:38.488005 5751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:38.497246 5751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:38.498436 5751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:38.499706 5751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:38.512526 5756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:38.512603 5759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:38.512573 5757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:38.517397 5751 server_base.cc:1034] running on GCE node
I20260807 04:22:38.518002 5751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:38.518903 5751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:38.520121 5751 hybrid_clock.cc:648] HybridClock initialized: now 1786076558520085 us; error 55 us; skew 500 ppm
I20260807 04:22:38.520430 5751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:38 dist-test-slave-zkpd krb5kdc[5638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076558, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:38.523630 5751 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:38.525173 5751 webserver.cc:460] Webserver started at http://127.1.244.193:45615/ using document root <none> and password file <none>
I20260807 04:22:38.525864 5751 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:38.526016 5751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:38.526326 5751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:38.529196 5751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "2b980e6a740a44068996e0fbd73c5fed"
format_stamp: "Formatted at 2026-08-07 04:22:38 on dist-test-slave-zkpd"
server_key: "54c823d4d6b87c3e78cf8acfb31458ee"
server_key_iv: "06be49b75f0716f81f9d83877655a1dc"
server_key_version: "encryptionkey@0"
I20260807 04:22:38.529953 5751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "2b980e6a740a44068996e0fbd73c5fed"
format_stamp: "Formatted at 2026-08-07 04:22:38 on dist-test-slave-zkpd"
server_key: "54c823d4d6b87c3e78cf8acfb31458ee"
server_key_iv: "06be49b75f0716f81f9d83877655a1dc"
server_key_version: "encryptionkey@0"
I20260807 04:22:38.534850 5751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:22:38.538129 5767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:38.539505 5751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:22:38.539674 5751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "2b980e6a740a44068996e0fbd73c5fed"
format_stamp: "Formatted at 2026-08-07 04:22:38 on dist-test-slave-zkpd"
server_key: "54c823d4d6b87c3e78cf8acfb31458ee"
server_key_iv: "06be49b75f0716f81f9d83877655a1dc"
server_key_version: "encryptionkey@0"
I20260807 04:22:38.539875 5751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:38.561781 5751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:38.565637 5751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:38.565932 5751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:38.566833 5751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:38.568185 5751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:38.568262 5751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:38.568426 5751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:38.568490 5751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:38.600858 5751 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:39973
I20260807 04:22:38.600944 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:39973 every 8 connection(s)
I20260807 04:22:38.602151 5751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:38.605803 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5751
I20260807 04:22:38.606068 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:38.606918 2003 external_mini_cluster.cc:1428] Setting key 7ee209fefc92561452e5a0e5993e72c4
Aug 07 04:22:38 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076558, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:38.625420 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:38.604695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:43677 (local address 127.1.244.254:46219)
0807 04:22:38.605161 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:38.605167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:38.606358 (+ 1191us) server_negotiation.cc:407] Connection header received
0807 04:22:38.608699 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:38.608705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:38.608784 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:38.608905 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:38.613439 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:38.614178 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:38.615750 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:38.616222 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:38.616363 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:38.619733 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:38.619738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:38.619742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:38.619773 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:38.622327 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:38.623058 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:38.623064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:38.623067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:38.623192 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:38.623632 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:38.623637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:38.623641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:38.623987 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:38.624141 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:38.624828 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 04:22:38.625049 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260807 04:22:38.626317 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:38.605291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46219 (local address 127.1.244.193:43677)
0807 04:22:38.606180 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:38.606233 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:38.608401 (+ 2168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:38.612176 (+ 3775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:38.612185 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:38.612709 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:38.613258 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:38.613276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:38.614349 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:38.614353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:38.615595 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:38.615608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:38.616466 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:38.616473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:38.616711 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:38.617384 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:38.617414 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:38.619516 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:38.622480 (+ 2964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:38.622489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:38.622504 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:38.622936 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:38.623307 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:38.623312 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:38.623316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:38.623515 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:38.624113 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:38.624120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:38.624453 (+ 333us) client_negotiation.cc:769] Sending connection context
0807 04:22:38.624729 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 04:22:38.625247 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":320,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260807 04:22:38.627826 5883 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46219
I20260807 04:22:38.628196 5883 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
I20260807 04:22:38.628888 5883 heartbeater.cc:511] Master 127.1.244.254:46219 requested a full tablet report, sending...
I20260807 04:22:38.631398 5687 ts_manager.cc:194] Registered new tserver with Master: 2b980e6a740a44068996e0fbd73c5fed (127.1.244.193:39973)
I20260807 04:22:38.633566 5687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:43677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:38.654552 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46219
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:38.970450 5889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:38.970706 5889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:38.970891 5889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:38.977864 5889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:38.977959 5889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:38.978120 5889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:38.987143 5889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:38.988292 5889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:38.989500 5889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:39.002797 5895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:39.002970 5894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:39.003333 5897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:39.003757 5889 server_base.cc:1034] running on GCE node
I20260807 04:22:39.004453 5889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:39.005369 5889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:39.006599 5889 hybrid_clock.cc:648] HybridClock initialized: now 1786076559006550 us; error 38 us; skew 500 ppm
I20260807 04:22:39.006942 5889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:39.010144 5889 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:39.011579 5889 webserver.cc:460] Webserver started at http://127.1.244.194:43169/ using document root <none> and password file <none>
I20260807 04:22:39.012290 5889 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:39.012422 5889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:39.012725 5889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:39.015574 5889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "8c731de029b94f1ea88115c87e180286"
format_stamp: "Formatted at 2026-08-07 04:22:39 on dist-test-slave-zkpd"
server_key: "dc0a080f2805030add02ba21c587137d"
server_key_iv: "91504e7d4fc846d35f150caf214f2bfa"
server_key_version: "encryptionkey@0"
I20260807 04:22:39.016414 5889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "8c731de029b94f1ea88115c87e180286"
format_stamp: "Formatted at 2026-08-07 04:22:39 on dist-test-slave-zkpd"
server_key: "dc0a080f2805030add02ba21c587137d"
server_key_iv: "91504e7d4fc846d35f150caf214f2bfa"
server_key_version: "encryptionkey@0"
I20260807 04:22:39.021297 5889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:22:39.024652 5904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:39.026014 5889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:22:39.026183 5889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "8c731de029b94f1ea88115c87e180286"
format_stamp: "Formatted at 2026-08-07 04:22:39 on dist-test-slave-zkpd"
server_key: "dc0a080f2805030add02ba21c587137d"
server_key_iv: "91504e7d4fc846d35f150caf214f2bfa"
server_key_version: "encryptionkey@0"
I20260807 04:22:39.026392 5889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:39.063040 5889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:39.066989 5889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:39.067340 5889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:39.068336 5889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:39.069610 5889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:39.069718 5889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:39.069871 5889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:39.069929 5889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:39.103693 5889 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:34631
I20260807 04:22:39.103726 6024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:34631 every 8 connection(s)
I20260807 04:22:39.104996 5889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:39.113785 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 5889
I20260807 04:22:39.113996 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:39.114413 2003 external_mini_cluster.cc:1428] Setting key f6202225022f2920f728900befad3957
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:39.126573 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.107549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:55015 (local address 127.1.244.254:46219)
0807 04:22:39.107740 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.107745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.109020 (+ 1275us) server_negotiation.cc:407] Connection header received
0807 04:22:39.111032 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.111037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.111102 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.111188 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.112680 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.113357 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.115027 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.115562 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.115692 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.119692 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.119701 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.119708 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.119754 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.123335 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.124113 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.124121 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.124127 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.124286 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.124827 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.124830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.124832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.125111 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.125218 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.126155 (+ 937us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.126321 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:22:39.128127 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.108121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46219 (local address 127.1.244.194:55015)
0807 04:22:39.108845 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:39.108889 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:39.110757 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:39.111404 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:39.111423 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:39.111983 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:39.112533 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:39.112550 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.113507 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.113510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:39.114882 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:39.114895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.115769 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.115774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:39.116064 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.116808 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:39.116846 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:39.119461 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:39.123483 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:39.123489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:39.123502 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:39.123984 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:39.124428 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:39.124435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:39.124441 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:39.124706 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:39.125237 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:39.125246 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:39.125707 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 04:22:39.126036 (+ 329us) client_negotiation.cc:240] Negotiation successful
0807 04:22:39.126605 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":325,"threads_started":1}
I20260807 04:22:39.129578 6025 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46219
I20260807 04:22:39.129914 6025 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:39.130887 6025 heartbeater.cc:511] Master 127.1.244.254:46219 requested a full tablet report, sending...
I20260807 04:22:39.132429 5687 ts_manager.cc:194] Registered new tserver with Master: 8c731de029b94f1ea88115c87e180286 (127.1.244.194:34631)
I20260807 04:22:39.133225 5687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:55015
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:39.164208 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46219
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:39.488696 6030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:39.488963 6030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:39.489152 6030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:39.496510 6030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:39.496629 6030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:39.496809 6030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:39.506522 6030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:39.507718 6030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:39.509063 6030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:39.521811 6035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:39.521812 6036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:39.522394 6038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:39.523151 6030 server_base.cc:1034] running on GCE node
I20260807 04:22:39.523708 6030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:39.524623 6030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:39.525815 6030 hybrid_clock.cc:648] HybridClock initialized: now 1786076559525771 us; error 33 us; skew 500 ppm
I20260807 04:22:39.526108 6030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:39.533375 6030 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:39.534742 6030 webserver.cc:460] Webserver started at http://127.1.244.195:40823/ using document root <none> and password file <none>
I20260807 04:22:39.535382 6030 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:39.535522 6030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:39.535822 6030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:39.538595 6030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "75eda36e21724190bcd92346f2cec113"
format_stamp: "Formatted at 2026-08-07 04:22:39 on dist-test-slave-zkpd"
server_key: "a153e2ce786f77915570ee7d613cbfb6"
server_key_iv: "1c49cd1bff24b0461b5be739c173b21a"
server_key_version: "encryptionkey@0"
I20260807 04:22:39.539484 6030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "75eda36e21724190bcd92346f2cec113"
format_stamp: "Formatted at 2026-08-07 04:22:39 on dist-test-slave-zkpd"
server_key: "a153e2ce786f77915570ee7d613cbfb6"
server_key_iv: "1c49cd1bff24b0461b5be739c173b21a"
server_key_version: "encryptionkey@0"
I20260807 04:22:39.544618 6030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 04:22:39.547828 6046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:39.549047 6030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:22:39.549213 6030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "75eda36e21724190bcd92346f2cec113"
format_stamp: "Formatted at 2026-08-07 04:22:39 on dist-test-slave-zkpd"
server_key: "a153e2ce786f77915570ee7d613cbfb6"
server_key_iv: "1c49cd1bff24b0461b5be739c173b21a"
server_key_version: "encryptionkey@0"
I20260807 04:22:39.549407 6030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:39.583819 6030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:39.587589 6030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:39.587880 6030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:39.588881 6030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:39.590173 6030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:39.590277 6030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:39.590446 6030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:39.590521 6030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:39.623754 6030 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:36695
I20260807 04:22:39.623807 6164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:36695 every 8 connection(s)
I20260807 04:22:39.625036 6030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:39.633564 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6030
I20260807 04:22:39.633783 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:39.634235 2003 external_mini_cluster.cc:1428] Setting key 8b79c8e452455dbb7f5ac4574b16959c
I20260807 04:22:39.636606 5883 heartbeater.cc:503] Master 127.1.244.254:46219 was elected leader, sending a full tablet report...
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:39.645450 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.627620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58729 (local address 127.1.244.254:46219)
0807 04:22:39.628172 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.628177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.629108 (+ 931us) server_negotiation.cc:407] Connection header received
0807 04:22:39.631004 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.631010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.631094 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.631214 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.632743 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.633769 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.635534 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.636193 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.636328 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.640118 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.640123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.640126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.640156 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.642762 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.643440 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.643444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.643446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.643579 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.644025 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.644029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.644031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.644303 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.644486 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.645028 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.645179 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":298,"threads_started":1}
I20260807 04:22:39.646565 6168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.628245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46219 (local address 127.1.244.195:58729)
0807 04:22:39.628948 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:39.628992 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:39.630720 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:39.631460 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:39.631468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:39.632018 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:39.632570 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:39.632587 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.633960 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.633964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:39.635377 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:39.635398 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.636461 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.636470 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:39.636758 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.637449 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:39.637477 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:39.639896 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:39.642872 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:39.642878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:39.642889 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:39.643342 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:39.643669 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:39.643672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:39.643675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:39.643837 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:39.644448 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:39.644453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:39.644720 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 04:22:39.644925 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 04:22:39.645253 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":312,"threads_started":1}
I20260807 04:22:39.648000 6165 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46219
I20260807 04:22:39.648352 6165 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:39.649127 6165 heartbeater.cc:511] Master 127.1.244.254:46219 requested a full tablet report, sending...
I20260807 04:22:39.650511 5687 ts_manager.cc:194] Registered new tserver with Master: 75eda36e21724190bcd92346f2cec113 (127.1.244.195:36695)
I20260807 04:22:39.651223 5687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:58729
I20260807 04:22:39.661580 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:39.712132 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.697777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.1.244.193:39973)
0807 04:22:39.698399 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.698406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.698432 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:22:39.698521 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.698527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.698730 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.698870 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.700249 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.701232 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.702425 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.702974 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.703068 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.706767 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.706779 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.706791 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.706821 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.709686 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.710278 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.710284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.710287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.710408 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.710733 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.710736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.710738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.711058 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.711276 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.711507 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.711720 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":337,"threads_started":1}
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:39.728893 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.716386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.1.244.254:46219)
0807 04:22:39.716621 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.716627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.716809 (+ 182us) server_negotiation.cc:407] Connection header received
0807 04:22:39.716921 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.716925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.717000 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.717079 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.718197 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.718917 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.720124 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.720521 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.720611 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.724058 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.724063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.724065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.724095 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.726659 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.727225 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.727229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.727231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.727344 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.727633 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.727636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.727638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.727994 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.728178 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.728415 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.728569 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 04:22:39.729411 5687 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53584
I20260807 04:22:39.730379 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5751
I20260807 04:22:39.743238 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.731223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.1.244.193:39973)
0807 04:22:39.731407 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.731414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.731687 (+ 273us) server_negotiation.cc:407] Connection header received
0807 04:22:39.731880 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.731884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.732024 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.732133 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.733321 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.734021 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.735290 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.735693 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.735833 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.738002 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.738006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.738009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.738040 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.740716 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.741345 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.741355 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.741357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.741475 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.741889 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.741894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.741898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.742147 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.742397 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.742715 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.742945 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:22:39.743839 5877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:39.824811 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5751
I20260807 04:22:39.843956 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5889
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:39.860323 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.844834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.1.244.194:34631)
0807 04:22:39.845380 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.845387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.845415 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:22:39.845488 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.845494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.845701 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.845839 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.847164 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.848495 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.849725 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.850269 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.850363 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.854620 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.854638 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.854655 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.854697 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.857845 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.858408 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.858416 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.858418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.858544 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.858840 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.858843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.858844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.859165 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.859407 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.859623 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.859875 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":277,"threads_started":1}
I20260807 04:22:39.861635 6020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:39.943421 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5889
I20260807 04:22:39.961848 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6030
Aug 07 04:22:39 dist-test-slave-zkpd krb5kdc[5638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:39.978080 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:39.962829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.1.244.195:36695)
0807 04:22:39.963446 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:39.963453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:39.963483 (+ 30us) server_negotiation.cc:407] Connection header received
0807 04:22:39.963548 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:39.963554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:39.963753 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:39.963884 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:39.965115 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.966133 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.967351 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:39.967952 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:39.968093 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:39.972349 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:39.972361 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:39.972374 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:39.972405 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:39.975211 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.975937 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.975949 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.975955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.976104 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:39.976497 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:39.976503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:39.976506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:39.976844 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:39.977173 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:39.977468 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 04:22:39.977735 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":278,"threads_started":1}
I20260807 04:22:39.979271 6160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:40.063974 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6030
I20260807 04:22:40.082367 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 5653
I20260807 04:22:40.083564 5719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:40.185845 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 5653
2026-08-07T04:22:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3930 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:40 dist-test-slave-zkpd krb5kdc[6186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:40 dist-test-slave-zkpd krb5kdc[6186](info): set up 2 sockets
Aug 07 04:22:40 dist-test-slave-zkpd krb5kdc[6186](info): commencing operation
krb5kdc: starting...
W20260807 04:22:41.294750 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.009s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:41 dist-test-slave-zkpd krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:41.502784 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33877
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:33877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:41.823570 6200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:41.823827 6200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:41.824023 6200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:41.831061 6200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:41.831197 6200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:41.831280 6200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:41.831380 6200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:41.831470 6200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:41.840785 6200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:33877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:41.841991 6200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:41.843251 6200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:41.854809 6205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:41.854986 6208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:41.854833 6206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:41.858069 6200 server_base.cc:1034] running on GCE node
I20260807 04:22:41.859062 6200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:41.860899 6200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:41.862141 6200 hybrid_clock.cc:648] HybridClock initialized: now 1786076561862061 us; error 54 us; skew 500 ppm
I20260807 04:22:41.862543 6200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:41 dist-test-slave-zkpd krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:41.865913 6200 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:41.867066 6200 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:41.867198 6200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:41.867517 6200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:41.870362 6200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "c9d4e5afbb48443788084d1ed6319707"
format_stamp: "Formatted at 2026-08-07 04:22:41 on dist-test-slave-zkpd"
server_key: "8794583c0356534e732775c1176b29d5"
server_key_iv: "2507c9450b07189d678e22f8c39696c5"
server_key_version: "encryptionkey@0"
I20260807 04:22:41.871228 6200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "c9d4e5afbb48443788084d1ed6319707"
format_stamp: "Formatted at 2026-08-07 04:22:41 on dist-test-slave-zkpd"
server_key: "8794583c0356534e732775c1176b29d5"
server_key_iv: "2507c9450b07189d678e22f8c39696c5"
server_key_version: "encryptionkey@0"
I20260807 04:22:41.876024 6200 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 04:22:41.879276 6214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:41.880524 6200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:41.880697 6200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "c9d4e5afbb48443788084d1ed6319707"
format_stamp: "Formatted at 2026-08-07 04:22:41 on dist-test-slave-zkpd"
server_key: "8794583c0356534e732775c1176b29d5"
server_key_iv: "2507c9450b07189d678e22f8c39696c5"
server_key_version: "encryptionkey@0"
I20260807 04:22:41.880903 6200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:41.896315 6200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:41.900008 6200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:41.900275 6200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:41.917692 6200 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:33877
I20260807 04:22:41.917716 6268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:33877 every 8 connection(s)
I20260807 04:22:41.918526 6200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:41.922029 6269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:41.922189 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6200
I20260807 04:22:41.922426 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:41.923067 2003 external_mini_cluster.cc:1428] Setting key adbe7216297c7964590d5feb3d4103ff
I20260807 04:22:41.935392 6269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707: Bootstrap starting.
Aug 07 04:22:41 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:41.939428 6269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:41.940696 6269 log.cc:824] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:41.943380 6269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707: No bootstrap required, opened a new log
I20260807 04:22:41.947472 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:41.924981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.1.244.254:33877)
0807 04:22:41.925759 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:41.925770 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:41.925809 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:22:41.927763 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:41.927783 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:41.928195 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:41.928511 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:41.930311 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:41.931638 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:41.933522 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:41.934197 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:41.934345 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:41.939761 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:41.939769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:41.939789 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:41.939830 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:41.942954 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:41.943641 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:41.943651 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:41.943661 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:41.943914 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:41.944270 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:41.944275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:41.944278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:41.945001 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:41.945316 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:41.945673 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 04:22:41.946122 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":406,"threads_started":1}
I20260807 04:22:41.952150 6269 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d4e5afbb48443788084d1ed6319707" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33877 } }
I20260807 04:22:41.952487 6269 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:41.952601 6269 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9d4e5afbb48443788084d1ed6319707, State: Initialized, Role: FOLLOWER
I20260807 04:22:41.953126 6269 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d4e5afbb48443788084d1ed6319707" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33877 } }
I20260807 04:22:41.953336 6269 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:41.953475 6269 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:41.953639 6269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:41.955753 6269 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d4e5afbb48443788084d1ed6319707" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33877 } }
I20260807 04:22:41.956280 6269 leader_election.cc:302] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9d4e5afbb48443788084d1ed6319707; no voters:
I20260807 04:22:41.956804 6269 leader_election.cc:288] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:41.956951 6274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:41.957623 6274 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [term 1 LEADER]: Becoming Leader. State: Replica: c9d4e5afbb48443788084d1ed6319707, State: Running, Role: LEADER
I20260807 04:22:41.958138 6274 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d4e5afbb48443788084d1ed6319707" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33877 } }
I20260807 04:22:41.958793 6269 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:41.961742 6275 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9d4e5afbb48443788084d1ed6319707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d4e5afbb48443788084d1ed6319707" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33877 } } }
I20260807 04:22:41.961714 6276 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9d4e5afbb48443788084d1ed6319707. Latest consensus state: current_term: 1 leader_uuid: "c9d4e5afbb48443788084d1ed6319707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9d4e5afbb48443788084d1ed6319707" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33877 } } }
I20260807 04:22:41.962131 6275 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:41.962193 6276 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:41.963725 6281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:41.969587 6281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:41.977702 6281 catalog_manager.cc:1347] Generated new cluster ID: 9b6e5417bc38435188fc14bb37d34f2d
I20260807 04:22:41.977813 6281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:41.993233 6281 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:41.994503 6281 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:42.005823 6281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707: Generated new TSK 0
I20260807 04:22:42.006593 6281 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:42.064324 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33877
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:42.392053 6296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:42.392382 6296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:42.392565 6296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:42.399755 6296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:42.399907 6296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:42.400122 6296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:42.409600 6296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:42.410881 6296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:42.412222 6296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:42.425107 6302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:42.425184 6305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:42.425163 6303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:42.426375 6296 server_base.cc:1034] running on GCE node
I20260807 04:22:42.426929 6296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:42.427803 6296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:42.429036 6296 hybrid_clock.cc:648] HybridClock initialized: now 1786076562428992 us; error 41 us; skew 500 ppm
I20260807 04:22:42.429358 6296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:42 dist-test-slave-zkpd krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076562, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:42.432617 6296 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:42.433679 6296 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:42.433787 6296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:42.434046 6296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:42.436978 6296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "0b8bc14149ec4e029e97e80bd32b396f"
format_stamp: "Formatted at 2026-08-07 04:22:42 on dist-test-slave-zkpd"
server_key: "6ce6529ac81e6c4ef46cdbe19d5c6322"
server_key_iv: "f8dc4bfde9eb69b162d750daa81df2ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:42.437768 6296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "0b8bc14149ec4e029e97e80bd32b396f"
format_stamp: "Formatted at 2026-08-07 04:22:42 on dist-test-slave-zkpd"
server_key: "6ce6529ac81e6c4ef46cdbe19d5c6322"
server_key_iv: "f8dc4bfde9eb69b162d750daa81df2ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:42.442688 6296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:22:42.445984 6311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:42.447381 6296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:22:42.447497 6296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "0b8bc14149ec4e029e97e80bd32b396f"
format_stamp: "Formatted at 2026-08-07 04:22:42 on dist-test-slave-zkpd"
server_key: "6ce6529ac81e6c4ef46cdbe19d5c6322"
server_key_iv: "f8dc4bfde9eb69b162d750daa81df2ae"
server_key_version: "encryptionkey@0"
I20260807 04:22:42.447681 6296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:42.467298 6296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:42.471315 6296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:42.471524 6296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:42.472429 6296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:42.473637 6296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:42.473740 6296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:42.473822 6296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:42.473886 6296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:42.506642 6296 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:41085
I20260807 04:22:42.506727 6423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:41085 every 8 connection(s)
I20260807 04:22:42.507513 6296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:42.515983 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6296
I20260807 04:22:42.516202 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:42.516857 2003 external_mini_cluster.cc:1428] Setting key 46cc78b0e2344664de46f1cbb7764908
Aug 07 04:22:42 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076562, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:42.528486 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:42.510066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49905 (local address 127.1.244.254:33877)
0807 04:22:42.510496 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:42.510501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:42.511472 (+ 971us) server_negotiation.cc:407] Connection header received
0807 04:22:42.513340 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:42.513346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:42.513441 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:42.513561 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:42.515022 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:42.515771 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:42.517612 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:42.518139 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:42.518249 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:42.522378 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:42.522383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:42.522386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:42.522419 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:42.524946 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:42.525868 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:42.525872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:42.525875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:42.525999 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:42.526470 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:42.526473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:42.526475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:42.526859 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:42.527031 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:42.527671 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 04:22:42.528017 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":209,"threads_started":1}
I20260807 04:22:42.529047 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:42.510631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33877 (local address 127.1.244.193:49905)
0807 04:22:42.511310 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:42.511355 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:42.513072 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:42.513800 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:42.513808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:42.514303 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:42.514843 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:42.514860 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:42.515956 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:42.515964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:42.517417 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:42.517432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:42.518370 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:42.518377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:42.518701 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:42.519459 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:42.519480 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:42.522152 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:42.525087 (+ 2935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:42.525098 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:42.525117 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:42.525756 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:42.526105 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:42.526112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:42.526119 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:42.526361 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:42.527007 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:42.527013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:42.527295 (+ 282us) client_negotiation.cc:769] Sending connection context
0807 04:22:42.527505 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 04:22:42.527865 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":281,"threads_started":1}
I20260807 04:22:42.530547 6424 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33877
I20260807 04:22:42.530870 6424 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:42.531481 6424 heartbeater.cc:511] Master 127.1.244.254:33877 requested a full tablet report, sending...
I20260807 04:22:42.533496 6231 ts_manager.cc:194] Registered new tserver with Master: 0b8bc14149ec4e029e97e80bd32b396f (127.1.244.193:41085)
I20260807 04:22:42.534885 6231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:49905
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:42.567657 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33877
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:42.886318 6430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:42.886569 6430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:42.886734 6430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:42.893664 6430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:42.893798 6430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:42.893987 6430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:42.903196 6430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:42.904505 6430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:42.905755 6430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:42.921837 6435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:42.921918 6438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:42.922201 6436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:42.924007 6430 server_base.cc:1034] running on GCE node
I20260807 04:22:42.924746 6430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:42.925704 6430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:42.926954 6430 hybrid_clock.cc:648] HybridClock initialized: now 1786076562926903 us; error 41 us; skew 500 ppm
I20260807 04:22:42.927264 6430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:42 dist-test-slave-zkpd krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076562, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:42.930501 6430 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:42.931694 6430 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:42.931805 6430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:42.932180 6430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:42.934986 6430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "48c5689b364441bc85250afae9b4b2d9"
format_stamp: "Formatted at 2026-08-07 04:22:42 on dist-test-slave-zkpd"
server_key: "b80ff56c4f2f5173c6e45fc6f4a3a28a"
server_key_iv: "1384633cff2eca579c66f1ab360f72d2"
server_key_version: "encryptionkey@0"
I20260807 04:22:42.935806 6430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "48c5689b364441bc85250afae9b4b2d9"
format_stamp: "Formatted at 2026-08-07 04:22:42 on dist-test-slave-zkpd"
server_key: "b80ff56c4f2f5173c6e45fc6f4a3a28a"
server_key_iv: "1384633cff2eca579c66f1ab360f72d2"
server_key_version: "encryptionkey@0"
I20260807 04:22:42.940687 6430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:22:42.944088 6444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:42.945271 6430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:22:42.945458 6430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "48c5689b364441bc85250afae9b4b2d9"
format_stamp: "Formatted at 2026-08-07 04:22:42 on dist-test-slave-zkpd"
server_key: "b80ff56c4f2f5173c6e45fc6f4a3a28a"
server_key_iv: "1384633cff2eca579c66f1ab360f72d2"
server_key_version: "encryptionkey@0"
I20260807 04:22:42.945667 6430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:42.962560 6430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:42.966997 6430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:42.967262 6430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:42.968211 6430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:42.969514 6430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:42.969636 6430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:42.969774 6430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:42.969870 6430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:43.002589 6430 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:38211
I20260807 04:22:43.002673 6559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:38211 every 8 connection(s)
I20260807 04:22:43.003458 6430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:43.007685 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6430
I20260807 04:22:43.007923 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:43.008375 2003 external_mini_cluster.cc:1428] Setting key 9225df4665057b59ecce75ecde8988a0
Aug 07 04:22:43 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076562, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:43.025107 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.005993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:52693 (local address 127.1.244.254:33877)
0807 04:22:43.006219 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.006231 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.007269 (+ 1038us) server_negotiation.cc:407] Connection header received
0807 04:22:43.009288 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.009294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.009388 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.009504 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:43.010994 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.011661 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.013275 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.013720 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.013857 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.017709 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:43.017718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:43.017725 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:43.017769 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:43.021602 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.022454 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.022463 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.022469 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.022625 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.023015 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.023023 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.023028 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.023468 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:43.023632 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.024355 (+ 723us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.024616 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":66}
I20260807 04:22:43.025681 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.006425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33877 (local address 127.1.244.194:52693)
0807 04:22:43.007091 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.007135 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.008988 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.009728 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.009735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.010231 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:43.010796 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.010814 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.011821 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.011825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.013115 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.013129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.013954 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.013962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.014178 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.014871 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:43.014892 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:43.017490 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:43.021855 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:43.021861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:43.021872 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:43.022315 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:43.022730 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:43.022733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:43.022736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:43.022902 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:43.023600 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:43.023605 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:43.023951 (+ 346us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.024196 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.024578 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":268,"threads_started":1}
I20260807 04:22:43.027092 6560 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33877
I20260807 04:22:43.027427 6560 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:43.028298 6560 heartbeater.cc:511] Master 127.1.244.254:33877 requested a full tablet report, sending...
I20260807 04:22:43.030072 6231 ts_manager.cc:194] Registered new tserver with Master: 48c5689b364441bc85250afae9b4b2d9 (127.1.244.194:38211)
I20260807 04:22:43.030906 6231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:52693
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:43.058334 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33877
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:43.378996 6565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:43.379349 6565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:43.379534 6565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:43.386756 6565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:43.386875 6565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:43.387058 6565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:43.396723 6565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:43.397958 6565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:43.399266 6565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:43.412020 6572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:43.411989 6574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:43.412101 6570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:43.413010 6565 server_base.cc:1034] running on GCE node
I20260807 04:22:43.413592 6565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:43.414494 6565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:43.415686 6565 hybrid_clock.cc:648] HybridClock initialized: now 1786076563415624 us; error 56 us; skew 500 ppm
I20260807 04:22:43.416031 6565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:43 dist-test-slave-zkpd krb5kdc[6186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076563, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:43.419111 6565 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:43.420292 6565 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:43.420447 6565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:43.420795 6565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:43.423547 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "8befe075cc2b46699762c2dc6314ede2"
format_stamp: "Formatted at 2026-08-07 04:22:43 on dist-test-slave-zkpd"
server_key: "4461740454569bfa7c33d7d803622402"
server_key_iv: "1804a6947e85978702b247f497f48f53"
server_key_version: "encryptionkey@0"
I20260807 04:22:43.424425 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "8befe075cc2b46699762c2dc6314ede2"
format_stamp: "Formatted at 2026-08-07 04:22:43 on dist-test-slave-zkpd"
server_key: "4461740454569bfa7c33d7d803622402"
server_key_iv: "1804a6947e85978702b247f497f48f53"
server_key_version: "encryptionkey@0"
I20260807 04:22:43.429267 6565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:22:43.432741 6580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:43.434029 6565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:22:43.434197 6565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "8befe075cc2b46699762c2dc6314ede2"
format_stamp: "Formatted at 2026-08-07 04:22:43 on dist-test-slave-zkpd"
server_key: "4461740454569bfa7c33d7d803622402"
server_key_iv: "1804a6947e85978702b247f497f48f53"
server_key_version: "encryptionkey@0"
I20260807 04:22:43.434434 6565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:43.480103 6565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:43.484448 6565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:43.484758 6565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:43.485724 6565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:43.487030 6565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:43.487149 6565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:43.487282 6565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:43.487397 6565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:43.521414 6565 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41977
I20260807 04:22:43.521466 6694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41977 every 8 connection(s)
I20260807 04:22:43.522240 6565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:43.528049 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6565
I20260807 04:22:43.528256 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:43.528748 2003 external_mini_cluster.cc:1428] Setting key 6e4b5e2e7e7cb1d05619fdf229480e28
Aug 07 04:22:43 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076563, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:43.537616 6424 heartbeater.cc:503] Master 127.1.244.254:33877 was elected leader, sending a full tablet report...
I20260807 04:22:43.541424 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.524565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:41021 (local address 127.1.244.254:33877)
0807 04:22:43.524745 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.524749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.525843 (+ 1094us) server_negotiation.cc:407] Connection header received
0807 04:22:43.527696 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.527703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.527838 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.528001 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:43.529563 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.530332 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.531874 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.532328 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.532482 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.535720 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:43.535734 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:43.535738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:43.535770 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:43.538632 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.539480 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.539483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.539485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.539611 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.540020 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.540023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.540025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.540289 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:43.540396 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.541023 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.541177 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:22:43.542562 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.524961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33877 (local address 127.1.244.195:41021)
0807 04:22:43.525675 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.525716 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.527383 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.528258 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.528267 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.528799 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:43.529386 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.529404 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.530506 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.530510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.531720 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.531733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.532580 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.532587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.532836 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.533432 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:43.533457 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:43.535526 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:43.538747 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:43.538753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:43.538764 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:43.539356 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:43.539711 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:43.539714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:43.539716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:43.539879 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:43.540412 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:43.540417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:43.540711 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.540940 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.541290 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":267,"threads_started":1}
I20260807 04:22:43.544026 6695 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33877
I20260807 04:22:43.544334 6695 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:43.544973 6695 heartbeater.cc:511] Master 127.1.244.254:33877 requested a full tablet report, sending...
I20260807 04:22:43.546262 6230 ts_manager.cc:194] Registered new tserver with Master: 8befe075cc2b46699762c2dc6314ede2 (127.1.244.195:41977)
I20260807 04:22:43.547081 6230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:41021
I20260807 04:22:43.556032 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:22:43.573495 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.560654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.1.244.254:33877)
0807 04:22:43.560833 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.560840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.561257 (+ 417us) server_negotiation.cc:407] Connection header received
0807 04:22:43.561465 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.561470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.561526 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.561641 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:43.563188 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.563975 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.565205 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.565614 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.565748 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.568088 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:43.568095 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:43.568113 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:43.568143 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:43.570635 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.571255 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.571261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.571265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.571422 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.571828 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.571836 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.571840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.572374 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:43.572643 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.572957 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.573177 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:22:43.577462 6230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:43.584110 6230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:43.608011 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.595860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41085 (local address 127.0.0.1:43974)
0807 04:22:43.597880 (+ 2020us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.597897 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.598011 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.599015 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.599022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.599052 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:43.599477 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.599488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.601522 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.601527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.603994 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.604009 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.607014 (+ 3005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.607022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.607197 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.607249 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.607400 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.607581 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1855,"thread_start_us":832,"threads_started":1}
I20260807 04:22:43.608860 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.596485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.1.244.193:41085)
0807 04:22:43.597074 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.597082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.598208 (+ 1126us) server_negotiation.cc:407] Connection header received
0807 04:22:43.598308 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.598314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.598549 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.598731 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:43.599702 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.601327 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.604172 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.606519 (+ 2347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.606655 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.606886 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.608409 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.608558 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":778,"threads_started":1}
I20260807 04:22:43.608858 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.595359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.1.244.194:38211)
0807 04:22:43.596632 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.596641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.597376 (+ 735us) server_negotiation.cc:407] Connection header received
0807 04:22:43.597738 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.597759 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.598030 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.599240 (+ 1210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:43.600174 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.601624 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.605000 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.607478 (+ 2478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.607574 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.607748 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.608372 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.608543 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1110,"thread_start_us":964,"threads_started":1}
I20260807 04:22:43.609560 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.595120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38211 (local address 127.0.0.1:32808)
0807 04:22:43.597191 (+ 2071us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.597250 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.597458 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.599488 (+ 2030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.599493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.599520 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:43.599987 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.599999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.602767 (+ 2768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.602772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.604801 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.604817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.607957 (+ 3140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.607964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.608171 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.608201 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.609066 (+ 865us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.609196 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1754,"thread_start_us":2434,"threads_started":1}
I20260807 04:22:43.614007 6359 tablet_service.cc:1467] Processing CreateTablet for tablet 9eac3ce651ba45dbb36357e3e07da547 (DEFAULT_TABLE table=test-table [id=b495cd3cdcfa42f7976f38309dd375ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:43.614070 6494 tablet_service.cc:1467] Processing CreateTablet for tablet 9eac3ce651ba45dbb36357e3e07da547 (DEFAULT_TABLE table=test-table [id=b495cd3cdcfa42f7976f38309dd375ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:43.615351 6359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eac3ce651ba45dbb36357e3e07da547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:43.615394 6494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eac3ce651ba45dbb36357e3e07da547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:43.617929 6707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.597977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41977 (local address 127.0.0.1:34774)
0807 04:22:43.609626 (+ 11649us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.609735 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.609827 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.611078 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.611082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.611108 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:43.611287 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.611298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.613243 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.613248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.615040 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.615057 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.617185 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.617193 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.617365 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.617398 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.617491 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.617640 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11541,"thread_start_us":12600,"threads_started":1}
I20260807 04:22:43.618379 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.598367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.1.244.195:41977)
0807 04:22:43.610527 (+ 12160us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.610535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.610564 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:22:43.610668 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.610675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.610887 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.611089 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:43.611498 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.613082 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.615218 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.616716 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.616850 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.617080 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.617823 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.618013 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11993,"thread_start_us":11804,"threads_started":1}
I20260807 04:22:43.622195 6628 tablet_service.cc:1467] Processing CreateTablet for tablet 9eac3ce651ba45dbb36357e3e07da547 (DEFAULT_TABLE table=test-table [id=b495cd3cdcfa42f7976f38309dd375ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:43.623639 6628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eac3ce651ba45dbb36357e3e07da547. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:43.627636 6713 tablet_bootstrap.cc:492] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Bootstrap starting.
I20260807 04:22:43.629614 6714 tablet_bootstrap.cc:492] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Bootstrap starting.
I20260807 04:22:43.631445 6713 tablet_bootstrap.cc:654] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:43.633162 6714 tablet_bootstrap.cc:654] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:43.633041 6713 log.cc:824] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:43.634436 6714 log.cc:824] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:43.636585 6713 tablet_bootstrap.cc:492] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: No bootstrap required, opened a new log
I20260807 04:22:43.636960 6713 ts_tablet_manager.cc:1397] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260807 04:22:43.636965 6717 tablet_bootstrap.cc:492] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Bootstrap starting.
I20260807 04:22:43.637539 6714 tablet_bootstrap.cc:492] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: No bootstrap required, opened a new log
I20260807 04:22:43.637806 6714 ts_tablet_manager.cc:1397] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.002s
I20260807 04:22:43.639997 6717 tablet_bootstrap.cc:654] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:43.640897 6717 log.cc:824] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:43.643208 6717 tablet_bootstrap.cc:492] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: No bootstrap required, opened a new log
I20260807 04:22:43.643689 6717 ts_tablet_manager.cc:1397] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260807 04:22:43.648360 6714 raft_consensus.cc:348] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.648366 6713 raft_consensus.cc:348] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.648797 6714 raft_consensus.cc:374] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:43.648800 6713 raft_consensus.cc:374] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:43.648918 6713 raft_consensus.cc:729] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5689b364441bc85250afae9b4b2d9, State: Initialized, Role: FOLLOWER
I20260807 04:22:43.648938 6714 raft_consensus.cc:729] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8bc14149ec4e029e97e80bd32b396f, State: Initialized, Role: FOLLOWER
I20260807 04:22:43.649566 6714 consensus_queue.cc:260] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.649544 6713 consensus_queue.cc:260] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.651084 6560 heartbeater.cc:503] Master 127.1.244.254:33877 was elected leader, sending a full tablet report...
I20260807 04:22:43.651736 6713 ts_tablet_manager.cc:1428] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260807 04:22:43.651564 6717 raft_consensus.cc:348] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.652051 6717 raft_consensus.cc:374] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:43.652192 6717 raft_consensus.cc:729] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8befe075cc2b46699762c2dc6314ede2, State: Initialized, Role: FOLLOWER
I20260807 04:22:43.652587 6714 ts_tablet_manager.cc:1428] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Time spent starting tablet: real 0.015s user 0.007s sys 0.006s
I20260807 04:22:43.654071 6717 consensus_queue.cc:260] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.655196 6695 heartbeater.cc:503] Master 127.1.244.254:33877 was elected leader, sending a full tablet report...
I20260807 04:22:43.657441 6717 ts_tablet_manager.cc:1428] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Time spent starting tablet: real 0.014s user 0.003s sys 0.007s
I20260807 04:22:43.722675 6720 raft_consensus.cc:482] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:43.723023 6720 raft_consensus.cc:504] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.724802 6720 leader_election.cc:288] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8befe075cc2b46699762c2dc6314ede2 (127.1.244.195:41977), 0b8bc14149ec4e029e97e80bd32b396f (127.1.244.193:41085)
I20260807 04:22:43.734331 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.725303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:34171 (local address 127.1.244.195:41977)
0807 04:22:43.725512 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.725520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.726352 (+ 832us) server_negotiation.cc:407] Connection header received
0807 04:22:43.726566 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.726570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.726643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.726745 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:43.727538 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.728713 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.731878 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.733127 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.733292 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.733367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.733947 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.734080 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:22:43.734333 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.725365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:53223 (local address 127.1.244.193:41085)
0807 04:22:43.725566 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.725573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.726352 (+ 779us) server_negotiation.cc:407] Connection header received
0807 04:22:43.726559 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.726564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.726643 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.726746 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:43.727574 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.728714 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.731878 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.733125 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.733292 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.733367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.733947 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.734080 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:22:43.734663 6723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.725188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41977 (local address 127.1.244.194:34171)
0807 04:22:43.726200 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.726224 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.726429 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.726775 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.726780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.726812 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:43.727334 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.727353 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.728864 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.728869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.731694 (+ 2825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.731711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.733431 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.733436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.733739 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.733792 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.733899 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.734080 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":355,"spinlock_wait_cycles":38912,"thread_start_us":556,"threads_started":1}
I20260807 04:22:43.734690 6722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.725282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41085 (local address 127.1.244.194:53223)
0807 04:22:43.726224 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:43.726247 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:43.726379 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:43.726775 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:43.726780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:43.726819 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:43.727410 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.727425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.728865 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.728869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.731694 (+ 2825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:43.731711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.733428 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.733436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:43.733719 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.733792 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 04:22:43.733899 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:22:43.734080 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":217,"spinlock_wait_cycles":3840,"thread_start_us":442,"threads_started":1}
I20260807 04:22:43.736002 6649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eac3ce651ba45dbb36357e3e07da547" candidate_uuid: "48c5689b364441bc85250afae9b4b2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8befe075cc2b46699762c2dc6314ede2" is_pre_election: true
I20260807 04:22:43.736002 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eac3ce651ba45dbb36357e3e07da547" candidate_uuid: "48c5689b364441bc85250afae9b4b2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8bc14149ec4e029e97e80bd32b396f" is_pre_election: true
I20260807 04:22:43.736599 6379 raft_consensus.cc:2454] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48c5689b364441bc85250afae9b4b2d9 in term 0.
I20260807 04:22:43.736599 6649 raft_consensus.cc:2454] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48c5689b364441bc85250afae9b4b2d9 in term 0.
I20260807 04:22:43.737346 6448 leader_election.cc:302] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b8bc14149ec4e029e97e80bd32b396f, 48c5689b364441bc85250afae9b4b2d9, 8befe075cc2b46699762c2dc6314ede2; no voters:
I20260807 04:22:43.737820 6720 raft_consensus.cc:2781] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:43.737987 6720 raft_consensus.cc:482] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:43.738106 6720 raft_consensus.cc:3037] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:43.740495 6720 raft_consensus.cc:504] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.741067 6720 leader_election.cc:288] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [CANDIDATE]: Term 1 election: Requested vote from peers 8befe075cc2b46699762c2dc6314ede2 (127.1.244.195:41977), 0b8bc14149ec4e029e97e80bd32b396f (127.1.244.193:41085)
I20260807 04:22:43.741498 6649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eac3ce651ba45dbb36357e3e07da547" candidate_uuid: "48c5689b364441bc85250afae9b4b2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8befe075cc2b46699762c2dc6314ede2"
I20260807 04:22:43.741516 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eac3ce651ba45dbb36357e3e07da547" candidate_uuid: "48c5689b364441bc85250afae9b4b2d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8bc14149ec4e029e97e80bd32b396f"
I20260807 04:22:43.741683 6649 raft_consensus.cc:3037] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:43.741714 6379 raft_consensus.cc:3037] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:43.743804 6379 raft_consensus.cc:2454] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48c5689b364441bc85250afae9b4b2d9 in term 1.
I20260807 04:22:43.743883 6649 raft_consensus.cc:2454] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48c5689b364441bc85250afae9b4b2d9 in term 1.
I20260807 04:22:43.744272 6446 leader_election.cc:302] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b8bc14149ec4e029e97e80bd32b396f, 48c5689b364441bc85250afae9b4b2d9; no voters:
I20260807 04:22:43.744647 6720 raft_consensus.cc:2781] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:43.745123 6720 raft_consensus.cc:686] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 1 LEADER]: Becoming Leader. State: Replica: 48c5689b364441bc85250afae9b4b2d9, State: Running, Role: LEADER
I20260807 04:22:43.745692 6720 consensus_queue.cc:237] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } }
I20260807 04:22:43.750207 6231 catalog_manager.cc:5495] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48c5689b364441bc85250afae9b4b2d9 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "48c5689b364441bc85250afae9b4b2d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c5689b364441bc85250afae9b4b2d9" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 } health_report { overall_health: UNKNOWN } } }
W20260807 04:22:43.755162 6561 tablet.cc:2369] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:22:43.759852 6425 tablet.cc:2369] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:22:43.773705 6696 tablet.cc:2369] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:22:43.795298 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.789990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.1.244.194:38211)
0807 04:22:43.790223 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.790231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.790254 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:22:43.790357 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.790361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.790436 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.790596 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:22:43.790969 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.791666 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.793110 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.793548 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.793699 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.794214 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:22:43.794684 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:22:43.794844 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.794939 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.795024 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:22:43.803345 6379 raft_consensus.cc:1261] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 1 FOLLOWER]: Refusing update from remote peer 48c5689b364441bc85250afae9b4b2d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:43.803345 6649 raft_consensus.cc:1261] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 1 FOLLOWER]: Refusing update from remote peer 48c5689b364441bc85250afae9b4b2d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:43.804347 6724 consensus_queue.cc:1035] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8befe075cc2b46699762c2dc6314ede2" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:43.804726 6720 consensus_queue.cc:1035] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b8bc14149ec4e029e97e80bd32b396f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:43.815243 6729 mvcc.cc:205] Tried to move back new op lower bound from 7315769605328154624 to 7315769605106905088. Current Snapshot: MvccSnapshot[applied={T|T < 7315769605328154624}]
I20260807 04:22:43.833811 6231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:43.834074 6231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:43.837527 6231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9d4e5afbb48443788084d1ed6319707: Sending DeleteTablet for 3 replicas of tablet 9eac3ce651ba45dbb36357e3e07da547
I20260807 04:22:43.838629 6494 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eac3ce651ba45dbb36357e3e07da547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:43 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:32808
I20260807 04:22:43.839031 6628 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eac3ce651ba45dbb36357e3e07da547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:43 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:34774
I20260807 04:22:43.839035 6359 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eac3ce651ba45dbb36357e3e07da547 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:43 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:43974
I20260807 04:22:43.844940 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6296
I20260807 04:22:43.848451 6737 tablet_replica.cc:330] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: stopping tablet replica
I20260807 04:22:43.849021 6737 raft_consensus.cc:2229] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:43.849910 6737 raft_consensus.cc:2258] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:43.852373 6737 ts_tablet_manager.cc:1905] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:43.861683 6737 ts_tablet_manager.cc:1918] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:43.861940 6737 log.cc:1196] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/9eac3ce651ba45dbb36357e3e07da547
I20260807 04:22:43.862738 6737 ts_tablet_manager.cc:1939] T 9eac3ce651ba45dbb36357e3e07da547 P 48c5689b364441bc85250afae9b4b2d9: Deleting consensus metadata
I20260807 04:22:43.864694 6218 catalog_manager.cc:4841] TS 48c5689b364441bc85250afae9b4b2d9 (127.1.244.194:38211): tablet 9eac3ce651ba45dbb36357e3e07da547 (table test-table [id=b495cd3cdcfa42f7976f38309dd375ae]) successfully deleted
I20260807 04:22:43.865624 6738 tablet_replica.cc:330] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: stopping tablet replica
I20260807 04:22:43.866137 6738 raft_consensus.cc:2229] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:43.866813 6738 raft_consensus.cc:2258] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:43.867435 6739 tablet_replica.cc:330] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: stopping tablet replica
I20260807 04:22:43.867826 6739 raft_consensus.cc:2229] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:43.868400 6739 raft_consensus.cc:2258] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:43.870149 6738 ts_tablet_manager.cc:1905] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:43.870434 6739 ts_tablet_manager.cc:1905] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 04:22:43 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:43.877203 6738 ts_tablet_manager.cc:1918] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:43.877378 6738 log.cc:1196] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/9eac3ce651ba45dbb36357e3e07da547
I20260807 04:22:43.877920 6738 ts_tablet_manager.cc:1939] T 9eac3ce651ba45dbb36357e3e07da547 P 8befe075cc2b46699762c2dc6314ede2: Deleting consensus metadata
I20260807 04:22:43.878876 6739 ts_tablet_manager.cc:1918] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:43.879086 6739 log.cc:1196] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/9eac3ce651ba45dbb36357e3e07da547
I20260807 04:22:43.879660 6739 ts_tablet_manager.cc:1939] T 9eac3ce651ba45dbb36357e3e07da547 P 0b8bc14149ec4e029e97e80bd32b396f: Deleting consensus metadata
I20260807 04:22:43.879801 6218 catalog_manager.cc:4841] TS 8befe075cc2b46699762c2dc6314ede2 (127.1.244.195:41977): tablet 9eac3ce651ba45dbb36357e3e07da547 (table test-table [id=b495cd3cdcfa42f7976f38309dd375ae]) successfully deleted
I20260807 04:22:43.881290 6216 catalog_manager.cc:4841] TS 0b8bc14149ec4e029e97e80bd32b396f (127.1.244.193:41085): tablet 9eac3ce651ba45dbb36357e3e07da547 (table test-table [id=b495cd3cdcfa42f7976f38309dd375ae]) successfully deleted
I20260807 04:22:43.883298 6710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.846406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.1.244.193:41085)
0807 04:22:43.846577 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.846583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.864434 (+ 17851us) server_negotiation.cc:407] Connection header received
0807 04:22:43.864909 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.864915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.865003 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.865085 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:43.866946 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.868201 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.870258 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.870910 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.871036 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:43.876376 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:43.876389 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:43.876412 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:43.876451 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:43.880486 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.881302 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.881312 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.881319 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.881486 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:43.881950 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:43.881954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:43.881956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:43.882295 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:43.882528 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:43.882861 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 04:22:43.883021 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:22:43.884265 6419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:43.974854 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6296
I20260807 04:22:43.995071 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6430
Aug 07 04:22:43 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:44.011430 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:43.995996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.1.244.194:38211)
0807 04:22:43.996188 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:43.996195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:43.996215 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:22:43.996313 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:43.996318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:43.996418 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:43.996527 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:43.997784 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:43.998604 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:43.999949 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:44.000498 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:44.000652 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:44.004792 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:44.004804 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:44.004816 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:44.004852 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:44.008635 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:44.009288 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:44.009304 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:44.009307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:44.009452 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:44.009823 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:44.009831 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:44.009838 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:44.010204 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:44.010539 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:44.010792 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 04:22:44.011017 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:22:44.012192 6555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:44.124898 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6430
I20260807 04:22:44.143517 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6565
Aug 07 04:22:44 dist-test-slave-zkpd krb5kdc[6186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:44.158264 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:44.144358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.1.244.195:41977)
0807 04:22:44.144552 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:44.144557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:44.144576 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:22:44.144717 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:44.144720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:44.144787 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:44.144871 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:44.146145 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:44.146875 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:44.148269 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:44.148642 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:44.148735 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:44.153011 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:44.153026 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:44.153042 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:44.153080 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:44.155644 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:44.156348 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:44.156356 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:44.156358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:44.156510 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:44.156843 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:44.156846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:44.156848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:44.157201 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:44.157419 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:44.157634 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 04:22:44.157840 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:22:44.159013 6690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:44.247474 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6565
I20260807 04:22:44.268311 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6200
I20260807 04:22:44.269451 6263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:44.377641 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6200
2026-08-07T04:22:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4192 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T04:22:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:44Z Disabled control of system clock
I20260807 04:22:44.450202 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41351
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 04:22:44.773765 6748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:44.774029 6748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:44.774111 6748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:44.781034 6748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:44.781134 6748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:44.781172 6748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:44.781230 6748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:44.781275 6748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:44.789981 6748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41351
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:44.791117 6748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:44.792799 6748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:44.804283 6753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:44.804356 6756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:44.804435 6754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:44.805747 6748 server_base.cc:1034] running on GCE node
I20260807 04:22:44.806409 6748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:44.807616 6748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:44.808815 6748 hybrid_clock.cc:648] HybridClock initialized: now 1786076564808754 us; error 62 us; skew 500 ppm
I20260807 04:22:44.809106 6748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:44.814432 6748 webserver.cc:460] Webserver started at http://127.1.244.254:44061/ using document root <none> and password file <none>
I20260807 04:22:44.815052 6748 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:44.815136 6748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:44.815402 6748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:44.818259 6748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "6f61562d481749caa0de6c5a8497c09e"
format_stamp: "Formatted at 2026-08-07 04:22:44 on dist-test-slave-zkpd"
server_key: "5f6236642e232716373619bf2ac45e6e"
server_key_iv: "2dcc3922274ce6c8ecdce1d78e79c054"
server_key_version: "encryptionkey@0"
I20260807 04:22:44.818974 6748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "6f61562d481749caa0de6c5a8497c09e"
format_stamp: "Formatted at 2026-08-07 04:22:44 on dist-test-slave-zkpd"
server_key: "5f6236642e232716373619bf2ac45e6e"
server_key_iv: "2dcc3922274ce6c8ecdce1d78e79c054"
server_key_version: "encryptionkey@0"
I20260807 04:22:44.823734 6748 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260807 04:22:44.827014 6762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:44.828374 6748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:44.828511 6748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "6f61562d481749caa0de6c5a8497c09e"
format_stamp: "Formatted at 2026-08-07 04:22:44 on dist-test-slave-zkpd"
server_key: "5f6236642e232716373619bf2ac45e6e"
server_key_iv: "2dcc3922274ce6c8ecdce1d78e79c054"
server_key_version: "encryptionkey@0"
I20260807 04:22:44.828682 6748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:44.834012 6748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:44.834689 6748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:44.834928 6748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:44.853228 6748 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41351
I20260807 04:22:44.853269 6814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41351 every 8 connection(s)
I20260807 04:22:44.854568 6748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:44.858067 6815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:44.858860 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6748
I20260807 04:22:44.859068 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:44.859711 2003 external_mini_cluster.cc:1428] Setting key 75481c4e04090d3c1d1c339500ee7444
I20260807 04:22:44.868047 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:44.861744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.1.244.254:41351)
0807 04:22:44.862948 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:44.862961 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:44.863003 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:22:44.864565 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:44.864586 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:44.864976 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:44.865293 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:44.866415 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:44.866422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:44.866434 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:44.866447 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:44.866507 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:44.866523 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:44.866684 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:44.866830 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 04:22:44.867082 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":537,"threads_started":1}
I20260807 04:22:44.871222 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e: Bootstrap starting.
I20260807 04:22:44.874184 6815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:44.875176 6815 log.cc:824] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:44.877928 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e: No bootstrap required, opened a new log
I20260807 04:22:44.885288 6815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f61562d481749caa0de6c5a8497c09e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41351 } }
I20260807 04:22:44.885648 6815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:44.885757 6815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f61562d481749caa0de6c5a8497c09e, State: Initialized, Role: FOLLOWER
I20260807 04:22:44.886300 6815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f61562d481749caa0de6c5a8497c09e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41351 } }
I20260807 04:22:44.886550 6815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:44.886700 6815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:44.886888 6815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:44.889041 6815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f61562d481749caa0de6c5a8497c09e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41351 } }
I20260807 04:22:44.889577 6815 leader_election.cc:302] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f61562d481749caa0de6c5a8497c09e; no voters:
I20260807 04:22:44.890110 6815 leader_election.cc:288] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:44.890233 6820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:44.891336 6820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [term 1 LEADER]: Becoming Leader. State: Replica: 6f61562d481749caa0de6c5a8497c09e, State: Running, Role: LEADER
I20260807 04:22:44.891827 6820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f61562d481749caa0de6c5a8497c09e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41351 } }
I20260807 04:22:44.892230 6815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:44.894433 6822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f61562d481749caa0de6c5a8497c09e. Latest consensus state: current_term: 1 leader_uuid: "6f61562d481749caa0de6c5a8497c09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f61562d481749caa0de6c5a8497c09e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41351 } } }
I20260807 04:22:44.894815 6822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:44.895541 6821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f61562d481749caa0de6c5a8497c09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f61562d481749caa0de6c5a8497c09e" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41351 } } }
I20260807 04:22:44.896121 6821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:44.898168 6829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:44.903270 6829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:44.911412 6829 catalog_manager.cc:1347] Generated new cluster ID: a75a323fe6094d2f8cecc1c1ce13eabe
I20260807 04:22:44.911516 6829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:44.921340 6829 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:44.921773 6829 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:44.930421 6829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e: Generated new TSK 0
I20260807 04:22:44.931119 6829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:22:44.939742 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41351
--builtin_ntp_servers=127.1.244.212:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 04:22:45.270803 6840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:45.271049 6840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:45.271212 6840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:45.278252 6840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:45.278340 6840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:45.278527 6840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:45.287169 6840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:45.288439 6840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:45.290134 6840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:45.303470 6846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:45.303515 6847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:45.303624 6849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:45.304600 6840 server_base.cc:1034] running on GCE node
I20260807 04:22:45.305240 6840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:45.306071 6840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:45.307277 6840 hybrid_clock.cc:648] HybridClock initialized: now 1786076565307224 us; error 46 us; skew 500 ppm
I20260807 04:22:45.307606 6840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:45.309857 6840 webserver.cc:460] Webserver started at http://127.1.244.193:36047/ using document root <none> and password file <none>
I20260807 04:22:45.310561 6840 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:45.310678 6840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:45.310997 6840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:45.313902 6840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "a01fe93c00684198bf4a6e3983d450d7"
format_stamp: "Formatted at 2026-08-07 04:22:45 on dist-test-slave-zkpd"
server_key: "f054370a1cac51f87a9bfafba09c4dc3"
server_key_iv: "9777a1e92050fe5c74c69bd128958461"
server_key_version: "encryptionkey@0"
I20260807 04:22:45.314684 6840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "a01fe93c00684198bf4a6e3983d450d7"
format_stamp: "Formatted at 2026-08-07 04:22:45 on dist-test-slave-zkpd"
server_key: "f054370a1cac51f87a9bfafba09c4dc3"
server_key_iv: "9777a1e92050fe5c74c69bd128958461"
server_key_version: "encryptionkey@0"
I20260807 04:22:45.319661 6840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:22:45.323145 6855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:45.324565 6840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:45.324779 6840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "a01fe93c00684198bf4a6e3983d450d7"
format_stamp: "Formatted at 2026-08-07 04:22:45 on dist-test-slave-zkpd"
server_key: "f054370a1cac51f87a9bfafba09c4dc3"
server_key_iv: "9777a1e92050fe5c74c69bd128958461"
server_key_version: "encryptionkey@0"
I20260807 04:22:45.325012 6840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:45.330502 6840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:45.331205 6840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:45.331502 6840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:45.332419 6840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:45.333729 6840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:45.333813 6840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:45.333890 6840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:45.333945 6840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:45.366206 6840 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44565
I20260807 04:22:45.366286 6968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44565 every 8 connection(s)
I20260807 04:22:45.367502 6840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:45.372298 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6840
I20260807 04:22:45.372507 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:45.373178 2003 external_mini_cluster.cc:1428] Setting key da7e1d2036867bd250b1d0d18ab667e9
I20260807 04:22:45.375764 6971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:45.369943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:48539 (local address 127.1.244.254:41351)
0807 04:22:45.370357 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:45.370363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:45.371440 (+ 1077us) server_negotiation.cc:407] Connection header received
0807 04:22:45.373265 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:45.373273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:45.373346 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:45.373482 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:45.374933 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:45.374938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:45.374941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:45.374956 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:45.374998 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:45.375010 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:45.375145 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:45.375373 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 04:22:45.375456 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":200,"threads_started":1}
I20260807 04:22:45.376430 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41351
--builtin_ntp_servers=127.1.244.212:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 04:22:45.376663 6972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:45.370509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41351 (local address 127.1.244.193:48539)
0807 04:22:45.371299 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:45.371338 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:45.372977 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:45.373664 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:45.373671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:45.373790 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:45.374570 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:45.374573 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:45.374621 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:45.374756 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:45.375154 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:45.375165 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:45.375174 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 04:22:45.375450 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 04:22:45.375644 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":233,"threads_started":1}
I20260807 04:22:45.378130 6969 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41351
I20260807 04:22:45.378444 6969 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:45.379076 6969 heartbeater.cc:511] Master 127.1.244.254:41351 requested a full tablet report, sending...
I20260807 04:22:45.381158 6779 ts_manager.cc:194] Registered new tserver with Master: a01fe93c00684198bf4a6e3983d450d7 (127.1.244.193:44565)
W20260807 04:22:45.702541 6973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:45.702790 6973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:45.702876 6973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:45.709857 6973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:45.709949 6973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:45.710053 6973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:45.718569 6973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:45.719663 6973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:45.721417 6973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:45.734107 6978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:45.734133 6981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:45.734061 6979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:45.734848 6973 server_base.cc:1034] running on GCE node
I20260807 04:22:45.735357 6973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:45.736223 6973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:45.737419 6973 hybrid_clock.cc:648] HybridClock initialized: now 1786076565737376 us; error 29 us; skew 500 ppm
I20260807 04:22:45.737717 6973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:45.739769 6973 webserver.cc:460] Webserver started at http://127.1.244.194:39221/ using document root <none> and password file <none>
I20260807 04:22:45.740446 6973 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:45.740564 6973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:45.740886 6973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:45.743593 6973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "609da54b548148f1b5e06d7b323654e2"
format_stamp: "Formatted at 2026-08-07 04:22:45 on dist-test-slave-zkpd"
server_key: "1ea1a9519e1c92b28fe83ec6315487c2"
server_key_iv: "f3f3e019f9420be7745d9e442ed4dc41"
server_key_version: "encryptionkey@0"
I20260807 04:22:45.744396 6973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "609da54b548148f1b5e06d7b323654e2"
format_stamp: "Formatted at 2026-08-07 04:22:45 on dist-test-slave-zkpd"
server_key: "1ea1a9519e1c92b28fe83ec6315487c2"
server_key_iv: "f3f3e019f9420be7745d9e442ed4dc41"
server_key_version: "encryptionkey@0"
I20260807 04:22:45.749338 6973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 04:22:45.752573 6987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:45.753762 6973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 04:22:45.753930 6973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "609da54b548148f1b5e06d7b323654e2"
format_stamp: "Formatted at 2026-08-07 04:22:45 on dist-test-slave-zkpd"
server_key: "1ea1a9519e1c92b28fe83ec6315487c2"
server_key_iv: "f3f3e019f9420be7745d9e442ed4dc41"
server_key_version: "encryptionkey@0"
I20260807 04:22:45.754159 6973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:45.759621 6973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:45.760407 6973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:45.760680 6973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:45.761541 6973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:45.762835 6973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:45.762970 6973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:45.763094 6973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:45.763183 6973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:45.795698 6973 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45259
I20260807 04:22:45.795783 7101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45259 every 8 connection(s)
I20260807 04:22:45.797063 6973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:45.798440 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 6973
I20260807 04:22:45.798614 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:45.799186 2003 external_mini_cluster.cc:1428] Setting key 348b837bb436b898a5c214ec1b7eade8
I20260807 04:22:45.802145 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41351
--builtin_ntp_servers=127.1.244.212:41089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 04:22:45.805810 6971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:45.799767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:58521 (local address 127.1.244.254:41351)
0807 04:22:45.800037 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:45.800044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:45.801610 (+ 1566us) server_negotiation.cc:407] Connection header received
0807 04:22:45.803390 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:45.803396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:45.803462 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:45.803576 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:45.805038 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:45.805044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:45.805049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:45.805068 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:45.805107 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:45.805118 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:45.805244 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:45.805451 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 04:22:45.805519 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:22:45.806711 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:45.800392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41351 (local address 127.1.244.194:58521)
0807 04:22:45.801449 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:45.801491 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:45.803124 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:45.803794 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:45.803802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:45.803931 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:45.804680 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:45.804682 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:45.804718 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:45.804803 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:45.805216 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:45.805227 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:45.805235 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:22:45.805532 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 04:22:45.805782 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":301,"threads_started":1}
I20260807 04:22:45.808053 7102 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41351
I20260807 04:22:45.808359 7102 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:45.808935 7102 heartbeater.cc:511] Master 127.1.244.254:41351 requested a full tablet report, sending...
I20260807 04:22:45.810367 6779 ts_manager.cc:194] Registered new tserver with Master: 609da54b548148f1b5e06d7b323654e2 (127.1.244.194:45259)
W20260807 04:22:46.121577 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:46.121869 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:46.122038 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:46.129235 7105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:46.129366 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:46.129530 7105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:46.138419 7105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:46.139624 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:46.141379 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:46.154166 7110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:46.154275 7111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:46.154304 7113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:46.155151 7105 server_base.cc:1034] running on GCE node
I20260807 04:22:46.155759 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:46.156644 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:46.157910 7105 hybrid_clock.cc:648] HybridClock initialized: now 1786076566157820 us; error 54 us; skew 500 ppm
I20260807 04:22:46.158231 7105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:46.160485 7105 webserver.cc:460] Webserver started at http://127.1.244.195:46501/ using document root <none> and password file <none>
I20260807 04:22:46.161152 7105 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:46.161268 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:46.161584 7105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:46.164372 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "7874686dac5b4c6fafec5640bd99baab"
format_stamp: "Formatted at 2026-08-07 04:22:46 on dist-test-slave-zkpd"
server_key: "34fb41c8f16dfc26ffc3e21e805b5386"
server_key_iv: "0b4d2d52e33e930a9a228eef7dae59ef"
server_key_version: "encryptionkey@0"
I20260807 04:22:46.165146 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "7874686dac5b4c6fafec5640bd99baab"
format_stamp: "Formatted at 2026-08-07 04:22:46 on dist-test-slave-zkpd"
server_key: "34fb41c8f16dfc26ffc3e21e805b5386"
server_key_iv: "0b4d2d52e33e930a9a228eef7dae59ef"
server_key_version: "encryptionkey@0"
I20260807 04:22:46.169984 7105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:22:46.173216 7119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:46.174441 7105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:22:46.174611 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "7874686dac5b4c6fafec5640bd99baab"
format_stamp: "Formatted at 2026-08-07 04:22:46 on dist-test-slave-zkpd"
server_key: "34fb41c8f16dfc26ffc3e21e805b5386"
server_key_iv: "0b4d2d52e33e930a9a228eef7dae59ef"
server_key_version: "encryptionkey@0"
I20260807 04:22:46.174842 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:46.180285 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:46.180997 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:46.181280 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:46.182152 7105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:46.183485 7105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:46.183606 7105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:46.183743 7105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:46.183832 7105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:46.216188 7105 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:36423
I20260807 04:22:46.216223 7232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:36423 every 8 connection(s)
I20260807 04:22:46.217424 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:46.224011 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7105
I20260807 04:22:46.224223 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:46.224692 2003 external_mini_cluster.cc:1428] Setting key 1ed16be2db47d60cd5e9c834aa7179ac
I20260807 04:22:46.225466 6971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.219954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45053 (local address 127.1.244.254:41351)
0807 04:22:46.220174 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.220180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.221546 (+ 1366us) server_negotiation.cc:407] Connection header received
0807 04:22:46.223136 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.223140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.223190 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.223273 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.224769 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.224775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.224782 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.224804 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.224845 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.224859 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.224997 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.225160 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.225224 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:22:46.226462 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.220490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41351 (local address 127.1.244.195:45053)
0807 04:22:46.221357 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:46.221391 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:46.222909 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:46.223520 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:46.223529 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:46.223670 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:46.224465 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:46.224468 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:46.224510 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:46.224589 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:46.224958 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:46.224967 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:46.224972 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:22:46.225177 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 04:22:46.225426 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":275,"threads_started":1}
I20260807 04:22:46.227941 7233 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41351
I20260807 04:22:46.228245 7233 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:46.228792 7233 heartbeater.cc:511] Master 127.1.244.254:41351 requested a full tablet report, sending...
I20260807 04:22:46.230093 6779 ts_manager.cc:194] Registered new tserver with Master: 7874686dac5b4c6fafec5640bd99baab (127.1.244.195:36423)
I20260807 04:22:46.240139 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:22:46.246811 6971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.244412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.1.244.254:41351)
0807 04:22:46.244576 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.244580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.245114 (+ 534us) server_negotiation.cc:407] Connection header received
0807 04:22:46.245233 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.245236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.245284 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.245359 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.246206 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.246210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.246213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.246227 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.246265 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.246277 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.246370 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.246470 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.246514 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:22:46.250554 6779 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:22:46.258188 6779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:22:46.285486 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.270328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:36423 (local address 127.0.0.1:39194)
0807 04:22:46.281423 (+ 11095us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:46.281477 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:46.281653 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:46.282778 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:46.282785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:46.282878 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:46.283755 (+ 877us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:46.283760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:46.283795 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:46.283877 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:46.284881 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:46.284894 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:46.284905 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 04:22:46.285083 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 04:22:46.285152 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10767,"threads_started":1}
I20260807 04:22:46.286237 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.270583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.1.244.195:36423)
0807 04:22:46.282303 (+ 11720us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.282310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.282339 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:22:46.282397 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.282402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.282584 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.282751 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.284495 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.284506 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.284521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.284538 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.284610 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.284635 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.284814 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.285736 (+ 922us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.285841 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11552,"thread_start_us":11401,"threads_started":1}
I20260807 04:22:46.288465 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.271303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.1.244.194:45259)
0807 04:22:46.284638 (+ 13335us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.284648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.285256 (+ 608us) server_negotiation.cc:407] Connection header received
0807 04:22:46.285395 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.285402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.285607 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.285746 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.287573 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.287582 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.287596 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.287634 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.287686 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.287706 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.287923 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.287998 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.288085 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13122,"thread_start_us":13331,"threads_started":1}
I20260807 04:22:46.288477 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.270974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:45259 (local address 127.0.0.1:55030)
0807 04:22:46.285136 (+ 14162us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:46.285155 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:46.285265 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:46.286691 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:46.286695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:46.286727 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:46.287328 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:46.287334 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:46.287354 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:46.287401 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:46.287827 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:46.287831 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:46.287836 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:22:46.288000 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 04:22:46.288051 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14008,"thread_start_us":9936,"threads_started":1}
I20260807 04:22:46.290279 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.286003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44565 (local address 127.0.0.1:42578)
0807 04:22:46.286454 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:46.286470 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:46.286539 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:46.288298 (+ 1759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:46.288308 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:46.288331 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:46.288920 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:46.288923 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:46.288940 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:46.288985 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:46.289847 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:46.289853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:46.289861 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:22:46.289992 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 04:22:46.290044 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351}
I20260807 04:22:46.291049 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.286091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.1.244.193:44565)
0807 04:22:46.287810 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.287818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.287844 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:22:46.287932 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.287939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.288119 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.288626 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.289525 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.289532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.289544 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.289559 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.289608 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.289626 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.289862 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.290617 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.290734 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1550,"thread_start_us":1409,"threads_started":1}
I20260807 04:22:46.291067 7167 tablet_service.cc:1467] Processing CreateTablet for tablet 6e51648e5ff74448ac1e7d646e4ee76b (DEFAULT_TABLE table=test-table [id=54f17753a2c5475c8f965ced412bfcd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:46.292363 7035 tablet_service.cc:1467] Processing CreateTablet for tablet 6e51648e5ff74448ac1e7d646e4ee76b (DEFAULT_TABLE table=test-table [id=54f17753a2c5475c8f965ced412bfcd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:46.293408 7167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e51648e5ff74448ac1e7d646e4ee76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:46.293593 7035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e51648e5ff74448ac1e7d646e4ee76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:46.294283 6903 tablet_service.cc:1467] Processing CreateTablet for tablet 6e51648e5ff74448ac1e7d646e4ee76b (DEFAULT_TABLE table=test-table [id=54f17753a2c5475c8f965ced412bfcd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:22:46.295572 6903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e51648e5ff74448ac1e7d646e4ee76b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:46.305997 7250 tablet_bootstrap.cc:492] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Bootstrap starting.
I20260807 04:22:46.308077 7251 tablet_bootstrap.cc:492] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Bootstrap starting.
I20260807 04:22:46.309739 7250 tablet_bootstrap.cc:654] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:46.309832 7252 tablet_bootstrap.cc:492] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Bootstrap starting.
I20260807 04:22:46.311160 7250 log.cc:824] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:46.311939 7251 tablet_bootstrap.cc:654] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:46.312964 7252 tablet_bootstrap.cc:654] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:46.313169 7251 log.cc:824] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:46.314203 7250 tablet_bootstrap.cc:492] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: No bootstrap required, opened a new log
I20260807 04:22:46.314184 7252 log.cc:824] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:46.314620 7250 ts_tablet_manager.cc:1397] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 04:22:46.316270 7251 tablet_bootstrap.cc:492] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: No bootstrap required, opened a new log
I20260807 04:22:46.316529 7251 ts_tablet_manager.cc:1397] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 04:22:46.317452 7252 tablet_bootstrap.cc:492] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: No bootstrap required, opened a new log
I20260807 04:22:46.317726 7252 ts_tablet_manager.cc:1397] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:22:46.323750 7251 raft_consensus.cc:348] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.324164 7251 raft_consensus.cc:374] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:46.324302 7251 raft_consensus.cc:729] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7874686dac5b4c6fafec5640bd99baab, State: Initialized, Role: FOLLOWER
I20260807 04:22:46.324908 7251 consensus_queue.cc:260] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.326054 7233 heartbeater.cc:503] Master 127.1.244.254:41351 was elected leader, sending a full tablet report...
I20260807 04:22:46.326421 7250 raft_consensus.cc:348] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.326769 7251 ts_tablet_manager.cc:1428] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260807 04:22:46.326874 7250 raft_consensus.cc:374] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:46.326969 7250 raft_consensus.cc:729] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609da54b548148f1b5e06d7b323654e2, State: Initialized, Role: FOLLOWER
I20260807 04:22:46.327621 7250 consensus_queue.cc:260] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.328895 7102 heartbeater.cc:503] Master 127.1.244.254:41351 was elected leader, sending a full tablet report...
I20260807 04:22:46.329797 7250 ts_tablet_manager.cc:1428] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
I20260807 04:22:46.330796 7252 raft_consensus.cc:348] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.331234 7252 raft_consensus.cc:374] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:46.331372 7252 raft_consensus.cc:729] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a01fe93c00684198bf4a6e3983d450d7, State: Initialized, Role: FOLLOWER
I20260807 04:22:46.332015 7252 consensus_queue.cc:260] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.333323 6969 heartbeater.cc:503] Master 127.1.244.254:41351 was elected leader, sending a full tablet report...
I20260807 04:22:46.333981 7252 ts_tablet_manager.cc:1428] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260807 04:22:46.369661 6970 tablet.cc:2369] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:22:46.373468 7257 raft_consensus.cc:482] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:46.373790 7257 raft_consensus.cc:504] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.375386 7257 leader_election.cc:288] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7874686dac5b4c6fafec5640bd99baab (127.1.244.195:36423), a01fe93c00684198bf4a6e3983d450d7 (127.1.244.193:44565)
I20260807 04:22:46.378988 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.375722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44459 (local address 127.1.244.195:36423)
0807 04:22:46.376009 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.376019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.376620 (+ 601us) server_negotiation.cc:407] Connection header received
0807 04:22:46.377033 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.377042 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.377130 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.377261 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.378324 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.378330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.378335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.378358 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.378390 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.378402 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.378529 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.378666 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.378728 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:22:46.379253 7259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.375711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:36423 (local address 127.1.244.194:44459)
0807 04:22:46.376482 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:46.376503 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:46.376650 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:46.377286 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:46.377290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:46.377310 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:46.378110 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:46.378112 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:46.378126 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:46.378167 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:46.378497 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:46.378509 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:46.378514 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:22:46.378668 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 04:22:46.378724 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":413,"threads_started":1}
I20260807 04:22:46.380185 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.376678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49417 (local address 127.1.244.193:44565)
0807 04:22:46.376916 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.376925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.378063 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 04:22:46.378230 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.378236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.378290 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.378388 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.379523 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.379529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.379537 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.379558 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.379596 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.379609 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.379691 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.379855 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.379942 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:22:46.380299 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.376597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44565 (local address 127.1.244.194:49417)
0807 04:22:46.377966 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:46.377980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:46.378104 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:46.378409 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:46.378414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:46.378443 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:46.379317 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:22:46.379319 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:46.379330 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:22:46.379390 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:46.379707 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:46.379714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:46.379719 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:22:46.379860 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 04:22:46.379954 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1260,"thread_start_us":999,"threads_started":1}
I20260807 04:22:46.380314 7188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e51648e5ff74448ac1e7d646e4ee76b" candidate_uuid: "609da54b548148f1b5e06d7b323654e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7874686dac5b4c6fafec5640bd99baab" is_pre_election: true
I20260807 04:22:46.380883 7188 raft_consensus.cc:2454] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609da54b548148f1b5e06d7b323654e2 in term 0.
I20260807 04:22:46.381138 6923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e51648e5ff74448ac1e7d646e4ee76b" candidate_uuid: "609da54b548148f1b5e06d7b323654e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01fe93c00684198bf4a6e3983d450d7" is_pre_election: true
I20260807 04:22:46.381524 6988 leader_election.cc:302] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609da54b548148f1b5e06d7b323654e2, 7874686dac5b4c6fafec5640bd99baab; no voters:
I20260807 04:22:46.381626 6923 raft_consensus.cc:2454] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609da54b548148f1b5e06d7b323654e2 in term 0.
I20260807 04:22:46.381946 7257 raft_consensus.cc:2781] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:22:46.382125 7257 raft_consensus.cc:482] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:22:46.382277 7257 raft_consensus.cc:3037] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:46.384765 7257 raft_consensus.cc:504] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.385250 7257 leader_election.cc:288] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [CANDIDATE]: Term 1 election: Requested vote from peers 7874686dac5b4c6fafec5640bd99baab (127.1.244.195:36423), a01fe93c00684198bf4a6e3983d450d7 (127.1.244.193:44565)
I20260807 04:22:46.385691 7188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e51648e5ff74448ac1e7d646e4ee76b" candidate_uuid: "609da54b548148f1b5e06d7b323654e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7874686dac5b4c6fafec5640bd99baab"
I20260807 04:22:46.385782 6923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e51648e5ff74448ac1e7d646e4ee76b" candidate_uuid: "609da54b548148f1b5e06d7b323654e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a01fe93c00684198bf4a6e3983d450d7"
I20260807 04:22:46.385897 7188 raft_consensus.cc:3037] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:46.386021 6923 raft_consensus.cc:3037] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:46.388289 7188 raft_consensus.cc:2454] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609da54b548148f1b5e06d7b323654e2 in term 1.
I20260807 04:22:46.388319 6923 raft_consensus.cc:2454] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609da54b548148f1b5e06d7b323654e2 in term 1.
I20260807 04:22:46.388684 6988 leader_election.cc:302] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609da54b548148f1b5e06d7b323654e2, 7874686dac5b4c6fafec5640bd99baab; no voters:
I20260807 04:22:46.388971 7257 raft_consensus.cc:2781] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:46.389400 7257 raft_consensus.cc:686] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 1 LEADER]: Becoming Leader. State: Replica: 609da54b548148f1b5e06d7b323654e2, State: Running, Role: LEADER
I20260807 04:22:46.389838 7257 consensus_queue.cc:237] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } }
I20260807 04:22:46.394644 6779 catalog_manager.cc:5495] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 609da54b548148f1b5e06d7b323654e2 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "609da54b548148f1b5e06d7b323654e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "609da54b548148f1b5e06d7b323654e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45259 } health_report { overall_health: HEALTHY } } }
I20260807 04:22:46.415385 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.413075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55034 (local address 127.1.244.194:45259)
0807 04:22:46.413247 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.413255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.413344 (+ 89us) server_negotiation.cc:407] Connection header received
0807 04:22:46.413575 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.413580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.413652 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.413795 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.414735 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.414740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.414745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.414763 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.414804 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.414816 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.414956 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.415066 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.415122 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:22:46.422870 6923 raft_consensus.cc:1261] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 1 FOLLOWER]: Refusing update from remote peer 609da54b548148f1b5e06d7b323654e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:46.422950 7188 raft_consensus.cc:1261] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 1 FOLLOWER]: Refusing update from remote peer 609da54b548148f1b5e06d7b323654e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:22:46.424002 7261 consensus_queue.cc:1035] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a01fe93c00684198bf4a6e3983d450d7" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:46.424428 7257 consensus_queue.cc:1035] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7874686dac5b4c6fafec5640bd99baab" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 36423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:46.437974 7265 mvcc.cc:205] Tried to move back new op lower bound from 7315769616059441152 to 7315769615936712704. Current Snapshot: MvccSnapshot[applied={T|T < 7315769616059441152}]
I20260807 04:22:46.439559 7264 mvcc.cc:205] Tried to move back new op lower bound from 7315769616059441152 to 7315769615936712704. Current Snapshot: MvccSnapshot[applied={T|T < 7315769616059441152}]
I20260807 04:22:46.445298 7266 mvcc.cc:205] Tried to move back new op lower bound from 7315769616059441152 to 7315769615936712704. Current Snapshot: MvccSnapshot[applied={T|T < 7315769616059441152}]
I20260807 04:22:46.453931 6778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:46.454273 6778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:22:46.459460 6778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f61562d481749caa0de6c5a8497c09e: Sending DeleteTablet for 3 replicas of tablet 6e51648e5ff74448ac1e7d646e4ee76b
I20260807 04:22:46.460857 7167 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e51648e5ff74448ac1e7d646e4ee76b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:46 UTC) from {username='slave'} at 127.0.0.1:39194
I20260807 04:22:46.461109 7035 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e51648e5ff74448ac1e7d646e4ee76b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:46 UTC) from {username='slave'} at 127.0.0.1:55030
I20260807 04:22:46.462392 6903 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e51648e5ff74448ac1e7d646e4ee76b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:22:46 UTC) from {username='slave'} at 127.0.0.1:42578
I20260807 04:22:46.463289 7275 tablet_replica.cc:330] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: stopping tablet replica
I20260807 04:22:46.463689 7275 raft_consensus.cc:2229] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:46.464236 7275 raft_consensus.cc:2258] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:46.464830 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6840
I20260807 04:22:46.467048 7275 ts_tablet_manager.cc:1905] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:46.477842 7275 ts_tablet_manager.cc:1918] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:46.478008 7275 log.cc:1196] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/6e51648e5ff74448ac1e7d646e4ee76b
I20260807 04:22:46.478560 7275 ts_tablet_manager.cc:1939] T 6e51648e5ff74448ac1e7d646e4ee76b P a01fe93c00684198bf4a6e3983d450d7: Deleting consensus metadata
I20260807 04:22:46.478922 7274 tablet_replica.cc:330] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: stopping tablet replica
I20260807 04:22:46.479568 7274 raft_consensus.cc:2229] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:22:46.480573 6763 catalog_manager.cc:4841] TS a01fe93c00684198bf4a6e3983d450d7 (127.1.244.193:44565): tablet 6e51648e5ff74448ac1e7d646e4ee76b (table test-table [id=54f17753a2c5475c8f965ced412bfcd6]) successfully deleted
I20260807 04:22:46.480796 7273 tablet_replica.cc:330] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: stopping tablet replica
I20260807 04:22:46.480861 7274 raft_consensus.cc:2258] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:46.481109 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.466293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.1.244.193:44565)
0807 04:22:46.466535 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.466544 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.478399 (+ 11855us) server_negotiation.cc:407] Connection header received
0807 04:22:46.478776 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.478781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.478889 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.478965 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.480313 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.480319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.480323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.480344 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.480379 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.480393 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.480480 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.480703 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.480775 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:22:46.481204 7273 raft_consensus.cc:2229] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:46.481725 7273 raft_consensus.cc:2258] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:46.481740 6964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:46.483206 7274 ts_tablet_manager.cc:1905] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:46.490510 7274 ts_tablet_manager.cc:1918] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:46.490691 7274 log.cc:1196] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/6e51648e5ff74448ac1e7d646e4ee76b
I20260807 04:22:46.491124 7273 ts_tablet_manager.cc:1905] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:46.491274 7274 ts_tablet_manager.cc:1939] T 6e51648e5ff74448ac1e7d646e4ee76b P 609da54b548148f1b5e06d7b323654e2: Deleting consensus metadata
I20260807 04:22:46.493008 6766 catalog_manager.cc:4841] TS 609da54b548148f1b5e06d7b323654e2 (127.1.244.194:45259): tablet 6e51648e5ff74448ac1e7d646e4ee76b (table test-table [id=54f17753a2c5475c8f965ced412bfcd6]) successfully deleted
I20260807 04:22:46.496639 7273 ts_tablet_manager.cc:1918] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:22:46.496811 7273 log.cc:1196] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/6e51648e5ff74448ac1e7d646e4ee76b
I20260807 04:22:46.497264 7273 ts_tablet_manager.cc:1939] T 6e51648e5ff74448ac1e7d646e4ee76b P 7874686dac5b4c6fafec5640bd99baab: Deleting consensus metadata
I20260807 04:22:46.498607 6763 catalog_manager.cc:4841] TS 7874686dac5b4c6fafec5640bd99baab (127.1.244.195:36423): tablet 6e51648e5ff74448ac1e7d646e4ee76b (table test-table [id=54f17753a2c5475c8f965ced412bfcd6]) successfully deleted
I20260807 04:22:46.585048 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6840
I20260807 04:22:46.611862 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6973
I20260807 04:22:46.614984 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.612738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.1.244.194:45259)
0807 04:22:46.612915 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.612920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.612979 (+ 59us) server_negotiation.cc:407] Connection header received
0807 04:22:46.613180 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.613184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.613235 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.613316 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.614272 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.614277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.614280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.614294 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.614325 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.614336 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.614446 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.614604 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.614649 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:22:46.615796 7097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:46.723060 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6973
I20260807 04:22:46.743224 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7105
I20260807 04:22:46.746039 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:46.744079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.1.244.195:36423)
0807 04:22:46.744260 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:46.744265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:46.744283 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:22:46.744428 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:46.744446 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:46.744535 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:46.744617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:46.745453 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:46.745457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:46.745461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:46.745474 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:46.745504 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:46.745516 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:46.745616 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:46.745679 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 04:22:46.745724 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:22:46.746665 7228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:46.833465 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7105
I20260807 04:22:46.852038 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 6748
I20260807 04:22:46.853298 6810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:46.980345 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 6748
2026-08-07T04:22:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2597 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 04:22:47.019979 2003 mini_oidc.cc:152] Starting JWKS server
I20260807 04:22:47.027132 2003 webserver.cc:460] Webserver started at https://127.0.0.1:43671/ using document root <none> and password file <none>
I20260807 04:22:47.027513 2003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 04:22:47.028224 2003 webserver.cc:460] Webserver started at http://127.0.0.1:38029/ using document root <none> and password file <none>
2026-08-07T04:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:47Z Disabled control of system clock
I20260807 04:22:47.058277 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36825
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36825
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/testchainca.cert with env {}
W20260807 04:22:47.382097 7286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:47.382426 7286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:47.382589 7286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:47.389691 7286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:47.389838 7286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:47.389909 7286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:47.389991 7286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:47.390058 7286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:22:47.390151 7286 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:22:47.390231 7286 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43671/jwks.json
I20260807 04:22:47.399771 7286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36825
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:47.401050 7286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:47.402288 7286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:47.413643 7291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:47.413702 7292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:47.413765 7294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:47.414494 7286 server_base.cc:1034] running on GCE node
I20260807 04:22:47.415217 7286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:47.416651 7286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:47.417884 7286 hybrid_clock.cc:648] HybridClock initialized: now 1786076567417840 us; error 39 us; skew 500 ppm
I20260807 04:22:47.418228 7286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:47.420508 7286 webserver.cc:460] Webserver started at http://127.1.244.254:35981/ using document root <none> and password file <none>
I20260807 04:22:47.421175 7286 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:47.421321 7286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:47.421633 7286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:47.424402 7286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "f91606f165844bac9009578843c04d7a"
format_stamp: "Formatted at 2026-08-07 04:22:47 on dist-test-slave-zkpd"
server_key: "7ec52ed606186915d72050ba7967716b"
server_key_iv: "24d9c24dc0a0c83786c74d4a2d69317c"
server_key_version: "encryptionkey@0"
I20260807 04:22:47.425237 7286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "f91606f165844bac9009578843c04d7a"
format_stamp: "Formatted at 2026-08-07 04:22:47 on dist-test-slave-zkpd"
server_key: "7ec52ed606186915d72050ba7967716b"
server_key_iv: "24d9c24dc0a0c83786c74d4a2d69317c"
server_key_version: "encryptionkey@0"
I20260807 04:22:47.430210 7286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:22:47.433494 7300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:47.434863 7286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:22:47.435065 7286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "f91606f165844bac9009578843c04d7a"
format_stamp: "Formatted at 2026-08-07 04:22:47 on dist-test-slave-zkpd"
server_key: "7ec52ed606186915d72050ba7967716b"
server_key_iv: "24d9c24dc0a0c83786c74d4a2d69317c"
server_key_version: "encryptionkey@0"
I20260807 04:22:47.435288 7286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:47.450582 7286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:47.451311 7286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:47.451603 7286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:47.473177 7286 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:36825
I20260807 04:22:47.473196 7351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:36825 every 8 connection(s)
I20260807 04:22:47.474552 7286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:47.476063 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7286
I20260807 04:22:47.476285 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:47.476936 2003 external_mini_cluster.cc:1428] Setting key 54ef04fc2c32433ffd0a7a90534d5b41
I20260807 04:22:47.478667 7352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:47.490518 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:47.478823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.1.244.254:36825)
0807 04:22:47.479776 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:47.479786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:47.479828 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:22:47.481448 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:47.481467 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:47.481899 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:47.482235 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:47.482953 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:47.484312 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:47.486204 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:47.486831 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:47.486954 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:47.488282 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:47.488290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:47.488309 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:47.488845 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:47.488932 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:47.488955 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:47.489038 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:47.489276 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 04:22:47.489617 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":345,"threads_started":1}
I20260807 04:22:47.491441 7352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a: Bootstrap starting.
I20260807 04:22:47.494114 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:22:47.494757 7352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:47.495918 7352 log.cc:824] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:47.498649 7352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a: No bootstrap required, opened a new log
I20260807 04:22:47.505446 7352 raft_consensus.cc:348] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91606f165844bac9009578843c04d7a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36825 } }
I20260807 04:22:47.505730 7352 raft_consensus.cc:374] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:47.505795 7352 raft_consensus.cc:729] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91606f165844bac9009578843c04d7a, State: Initialized, Role: FOLLOWER
I20260807 04:22:47.506331 7352 consensus_queue.cc:260] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91606f165844bac9009578843c04d7a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36825 } }
I20260807 04:22:47.506551 7352 raft_consensus.cc:388] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:47.506644 7352 raft_consensus.cc:482] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:47.506767 7352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:47.508747 7352 raft_consensus.cc:504] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91606f165844bac9009578843c04d7a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36825 } }
I20260807 04:22:47.509174 7352 leader_election.cc:302] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f91606f165844bac9009578843c04d7a; no voters:
I20260807 04:22:47.509811 7352 leader_election.cc:288] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:47.509955 7357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:47.510901 7357 raft_consensus.cc:686] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [term 1 LEADER]: Becoming Leader. State: Replica: f91606f165844bac9009578843c04d7a, State: Running, Role: LEADER
I20260807 04:22:47.511368 7357 consensus_queue.cc:237] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91606f165844bac9009578843c04d7a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36825 } }
I20260807 04:22:47.512120 7352 sys_catalog.cc:564] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:47.514671 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f91606f165844bac9009578843c04d7a. Latest consensus state: current_term: 1 leader_uuid: "f91606f165844bac9009578843c04d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91606f165844bac9009578843c04d7a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36825 } } }
I20260807 04:22:47.514519 7358 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f91606f165844bac9009578843c04d7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91606f165844bac9009578843c04d7a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36825 } } }
I20260807 04:22:47.515069 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:47.515069 7358 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:47.516888 7362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:47.522716 7362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:47.530933 7362 catalog_manager.cc:1347] Generated new cluster ID: aa668874c2104c43beca094ca3768f0f
I20260807 04:22:47.531044 7362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:47.558346 7362 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:47.560292 7362 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:47.578951 7362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f91606f165844bac9009578843c04d7a: Generated new TSK 0
I20260807 04:22:47.580116 7362 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 04:22:47.626588 7383 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 04:22:47.626915 7383 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 04:22:47.629168 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:47.603866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.1.244.254:36825)
0807 04:22:47.604127 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:47.604135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:47.604579 (+ 444us) server_negotiation.cc:407] Connection header received
0807 04:22:47.604964 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:47.604968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:47.605034 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:47.605161 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:22:47.605823 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:47.606562 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:47.608705 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:47.609177 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:47.609270 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:47.609893 (+ 623us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:22:47.628460 (+ 18567us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 04:22:47.628845 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 04:22:47.629483 7355 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 04:22:47.630817 7382 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:47.603748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36825 (local address 127.0.0.1:33070)
0807 04:22:47.604468 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:47.604489 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:47.604676 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:47.605225 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:47.605230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:47.605248 (+ 18us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:22:47.605662 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:47.605675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:47.606701 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:47.606707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:47.608564 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:47.608581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:47.609361 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:47.609368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:47.609571 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:47.609646 (+ 75us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:22:47.628760 (+ 19114us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 04:22:47.629196 (+ 436us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:36825: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":368,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.244.254:36825: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 04:22:47.633301 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7286
I20260807 04:22:47.634373 7347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:47.730731 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7286
2026-08-07T04:22:47Z chronyd exiting
I20260807 04:22:47.784605 2003 test_util.cc:175] -----------------------------------------------
I20260807 04:22:47.784755 2003 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786076526463677-2003-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (769 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 04:22:47.788630 2003 mini_oidc.cc:152] Starting JWKS server
I20260807 04:22:47.790371 2003 webserver.cc:460] Webserver started at https://127.0.0.1:45339/ using document root <none> and password file <none>
I20260807 04:22:47.790598 2003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 04:22:47.791219 2003 webserver.cc:460] Webserver started at http://127.0.0.1:43731/ using document root <none> and password file <none>
2026-08-07T04:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:47Z Disabled control of system clock
I20260807 04:22:47.819787 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36277
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:44101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36277
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45339/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/testchainca.cert with env {}
W20260807 04:22:48.139271 7390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:48.139547 7390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:48.139632 7390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:48.146878 7390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:48.146989 7390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:48.147033 7390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:48.147091 7390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:48.147130 7390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:22:48.147189 7390 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:22:48.147228 7390 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45339/jwks.json
I20260807 04:22:48.157109 7390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45339/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:48.158280 7390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:48.159519 7390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:48.171319 7395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:48.171360 7396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:48.171407 7399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:48.172302 7390 server_base.cc:1034] running on GCE node
I20260807 04:22:48.173084 7390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:48.174396 7390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:48.175624 7390 hybrid_clock.cc:648] HybridClock initialized: now 1786076568175583 us; error 52 us; skew 500 ppm
I20260807 04:22:48.176036 7390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:48.178246 7390 webserver.cc:460] Webserver started at http://127.1.244.254:43765/ using document root <none> and password file <none>
I20260807 04:22:48.178925 7390 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:48.179065 7390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:48.179389 7390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:48.182237 7390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "00d3d694d58b4ae899a489d5e50aaa80"
format_stamp: "Formatted at 2026-08-07 04:22:48 on dist-test-slave-zkpd"
server_key: "2ff6b2f3148d3ec0583d2c8dfd61282c"
server_key_iv: "a724d3e17f0136c1a25787ac23b37992"
server_key_version: "encryptionkey@0"
I20260807 04:22:48.183009 7390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "00d3d694d58b4ae899a489d5e50aaa80"
format_stamp: "Formatted at 2026-08-07 04:22:48 on dist-test-slave-zkpd"
server_key: "2ff6b2f3148d3ec0583d2c8dfd61282c"
server_key_iv: "a724d3e17f0136c1a25787ac23b37992"
server_key_version: "encryptionkey@0"
I20260807 04:22:48.187741 7390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 04:22:48.190892 7405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:48.192236 7390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:48.192440 7390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "00d3d694d58b4ae899a489d5e50aaa80"
format_stamp: "Formatted at 2026-08-07 04:22:48 on dist-test-slave-zkpd"
server_key: "2ff6b2f3148d3ec0583d2c8dfd61282c"
server_key_iv: "a724d3e17f0136c1a25787ac23b37992"
server_key_version: "encryptionkey@0"
I20260807 04:22:48.192672 7390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:48.205776 7390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:48.206496 7390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:48.206820 7390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:48.224799 7390 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:36277
I20260807 04:22:48.224820 7457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:36277 every 8 connection(s)
I20260807 04:22:48.226094 7390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:48.226646 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7390
I20260807 04:22:48.226850 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:48.227665 2003 external_mini_cluster.cc:1428] Setting key 05dc98d93ea714ea721706a7d74b0206
I20260807 04:22:48.230388 7458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:48.240897 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:48.229586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.1.244.254:36277)
0807 04:22:48.230560 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:48.230574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:48.230622 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:22:48.232377 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:48.232404 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:48.232826 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:48.233210 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:48.233786 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.235333 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.236702 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.237342 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.237537 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:48.238728 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:48.238735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:48.238747 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:48.239291 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:48.239366 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:48.239389 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:48.239523 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:48.239744 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 04:22:48.240071 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":334,"threads_started":1}
I20260807 04:22:48.241503 7458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80: Bootstrap starting.
I20260807 04:22:48.244520 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:22:48.244922 7458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:48.245900 7458 log.cc:824] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:48.248680 7458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80: No bootstrap required, opened a new log
I20260807 04:22:48.255656 7458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d3d694d58b4ae899a489d5e50aaa80" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36277 } }
I20260807 04:22:48.256047 7458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:48.256158 7458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d3d694d58b4ae899a489d5e50aaa80, State: Initialized, Role: FOLLOWER
I20260807 04:22:48.256719 7458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d3d694d58b4ae899a489d5e50aaa80" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36277 } }
I20260807 04:22:48.256950 7458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:48.257102 7458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:48.257275 7458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:48.259547 7458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d3d694d58b4ae899a489d5e50aaa80" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36277 } }
I20260807 04:22:48.260107 7458 leader_election.cc:302] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00d3d694d58b4ae899a489d5e50aaa80; no voters:
I20260807 04:22:48.260639 7458 leader_election.cc:288] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:48.260787 7463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:48.261526 7463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [term 1 LEADER]: Becoming Leader. State: Replica: 00d3d694d58b4ae899a489d5e50aaa80, State: Running, Role: LEADER
I20260807 04:22:48.262043 7463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d3d694d58b4ae899a489d5e50aaa80" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36277 } }
I20260807 04:22:48.262764 7458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:48.271036 7465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00d3d694d58b4ae899a489d5e50aaa80. Latest consensus state: current_term: 1 leader_uuid: "00d3d694d58b4ae899a489d5e50aaa80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d3d694d58b4ae899a489d5e50aaa80" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36277 } } }
I20260807 04:22:48.271497 7465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:48.271977 7464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00d3d694d58b4ae899a489d5e50aaa80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00d3d694d58b4ae899a489d5e50aaa80" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36277 } } }
I20260807 04:22:48.272414 7464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:48.272558 7472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:48.279913 7472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:48.290556 7472 catalog_manager.cc:1347] Generated new cluster ID: 226e21221f7c45ca9ddf8ccfbed1921d
I20260807 04:22:48.290674 7472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:48.318348 7472 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:48.320201 7472 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:48.331180 7472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00d3d694d58b4ae899a489d5e50aaa80: Generated new TSK 0
I20260807 04:22:48.332208 7472 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 04:22:48.383456 7489 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 04:22:48.383585 7489 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 04:22:48.384479 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:48.357168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.1.244.254:36277)
0807 04:22:48.357384 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:48.357391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:48.357929 (+ 538us) server_negotiation.cc:407] Connection header received
0807 04:22:48.358140 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:48.358143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:48.358211 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:48.358328 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:22:48.358867 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.359603 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.361607 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.362026 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.362114 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:48.362760 (+ 646us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:22:48.383939 (+ 21179us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45339/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 04:22:48.384250 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45339/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 04:22:48.384617 7461 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45339/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 04:22:48.384872 7488 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:48.357158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36277 (local address 127.0.0.1:40990)
0807 04:22:48.357817 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:48.357841 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:48.357986 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:48.358377 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:48.358382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:48.358395 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:22:48.358691 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:48.358703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.359737 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.359744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:48.361479 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:48.361495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.362184 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.362190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:48.362399 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:48.362445 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:22:48.384190 (+ 21745us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45339/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 04:22:48.384464 (+ 274us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:36277: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45339/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":340,"threads_started":1}
I20260807 04:22:48.386183 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7390
I20260807 04:22:48.387331 7453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:48.480916 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7390
2026-08-07T04:22:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (749 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 04:22:48.538350 2003 mini_oidc.cc:152] Starting JWKS server
I20260807 04:22:48.540282 2003 webserver.cc:460] Webserver started at https://127.0.0.1:37707/ using document root <none> and password file <none>
I20260807 04:22:48.540545 2003 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 04:22:48.541160 2003 webserver.cc:460] Webserver started at http://127.0.0.1:46359/ using document root <none> and password file <none>
2026-08-07T04:22:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:48Z Disabled control of system clock
I20260807 04:22:48.572688 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39561
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39561
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37707/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 04:22:48.886250 7496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:48.886556 7496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:48.886739 7496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:48.893656 7496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:48.893844 7496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:48.893932 7496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:48.894039 7496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:48.894124 7496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:22:48.894210 7496 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:22:48.894292 7496 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37707/jwks.json
I20260807 04:22:48.902997 7496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39561
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37707/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:48.904281 7496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:48.906461 7496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:48.918015 7504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:48.918110 7501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:48.918035 7502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:48.918820 7496 server_base.cc:1034] running on GCE node
I20260807 04:22:48.919524 7496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:48.920830 7496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:48.922031 7496 hybrid_clock.cc:648] HybridClock initialized: now 1786076568921983 us; error 34 us; skew 500 ppm
I20260807 04:22:48.922318 7496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:48.924487 7496 webserver.cc:460] Webserver started at http://127.1.244.254:35689/ using document root <none> and password file <none>
I20260807 04:22:48.925107 7496 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:48.925195 7496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:48.925444 7496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:48.928208 7496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "bfa23e336dec448da22c1641a065f77d"
format_stamp: "Formatted at 2026-08-07 04:22:48 on dist-test-slave-zkpd"
server_key: "9c8f05ad16abcd58531cfe189e9186bb"
server_key_iv: "29a0451b1c80ca6b7f5f0289b78163a2"
server_key_version: "encryptionkey@0"
I20260807 04:22:48.928946 7496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "bfa23e336dec448da22c1641a065f77d"
format_stamp: "Formatted at 2026-08-07 04:22:48 on dist-test-slave-zkpd"
server_key: "9c8f05ad16abcd58531cfe189e9186bb"
server_key_iv: "29a0451b1c80ca6b7f5f0289b78163a2"
server_key_version: "encryptionkey@0"
I20260807 04:22:48.933671 7496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:22:48.936929 7511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:48.938167 7496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:22:48.938345 7496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "bfa23e336dec448da22c1641a065f77d"
format_stamp: "Formatted at 2026-08-07 04:22:48 on dist-test-slave-zkpd"
server_key: "9c8f05ad16abcd58531cfe189e9186bb"
server_key_iv: "29a0451b1c80ca6b7f5f0289b78163a2"
server_key_version: "encryptionkey@0"
I20260807 04:22:48.938548 7496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:48.958480 7496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:48.959240 7496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:48.959586 7496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:48.977918 7496 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39561
I20260807 04:22:48.977941 7563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39561 every 8 connection(s)
I20260807 04:22:48.979152 7496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:48.980439 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7496
I20260807 04:22:48.980669 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:48.981498 2003 external_mini_cluster.cc:1428] Setting key b6a52f873c81e7727936d432b4bbac91
I20260807 04:22:48.983711 7564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:48.994415 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:48.983502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.1.244.254:39561)
0807 04:22:48.984479 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:48.984489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:48.984529 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:22:48.986010 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:48.986029 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:48.986489 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:48.986894 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:48.987441 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.988917 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.990257 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:48.990883 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:48.991075 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:48.992281 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:48.992289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:48.992300 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:48.992847 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:48.992914 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:48.992937 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:48.993066 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:48.993253 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 04:22:48.993541 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":407,"threads_started":1}
I20260807 04:22:48.994706 7564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d: Bootstrap starting.
I20260807 04:22:48.997999 7564 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:48.998055 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:22:48.998960 7564 log.cc:824] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:49.001993 7564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d: No bootstrap required, opened a new log
I20260807 04:22:49.009449 7564 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa23e336dec448da22c1641a065f77d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39561 } }
I20260807 04:22:49.009856 7564 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:49.009956 7564 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa23e336dec448da22c1641a065f77d, State: Initialized, Role: FOLLOWER
I20260807 04:22:49.010402 7564 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa23e336dec448da22c1641a065f77d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39561 } }
I20260807 04:22:49.010596 7564 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:49.010720 7564 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:49.010845 7564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:49.012869 7564 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa23e336dec448da22c1641a065f77d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39561 } }
I20260807 04:22:49.013304 7564 leader_election.cc:302] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfa23e336dec448da22c1641a065f77d; no voters:
I20260807 04:22:49.013828 7564 leader_election.cc:288] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:49.013984 7569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:49.014688 7569 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [term 1 LEADER]: Becoming Leader. State: Replica: bfa23e336dec448da22c1641a065f77d, State: Running, Role: LEADER
I20260807 04:22:49.015242 7569 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa23e336dec448da22c1641a065f77d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39561 } }
I20260807 04:22:49.015967 7564 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:49.018404 7571 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa23e336dec448da22c1641a065f77d. Latest consensus state: current_term: 1 leader_uuid: "bfa23e336dec448da22c1641a065f77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa23e336dec448da22c1641a065f77d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39561 } } }
I20260807 04:22:49.018347 7570 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfa23e336dec448da22c1641a065f77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa23e336dec448da22c1641a065f77d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39561 } } }
I20260807 04:22:49.018831 7571 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:49.018841 7570 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:49.022396 7576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:49.028713 7576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:49.037783 7576 catalog_manager.cc:1347] Generated new cluster ID: 59a1e4bd23a94c0789689ba942db1625
I20260807 04:22:49.037901 7576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:49.065686 7576 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:49.066905 7576 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:49.086143 7576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfa23e336dec448da22c1641a065f77d: Generated new TSK 0
I20260807 04:22:49.086933 7576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:22:49.114658 7567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:49.106673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.1.244.254:39561)
0807 04:22:49.106913 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:49.106919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:49.107293 (+ 374us) server_negotiation.cc:407] Connection header received
0807 04:22:49.107501 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:49.107505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:49.107572 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:49.107699 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:22:49.108290 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.108997 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.111023 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.111460 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.111542 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:49.112199 (+ 657us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:22:49.114031 (+ 1832us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37707/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 04:22:49.114357 (+ 326us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37707/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 04:22:49.114921 7593 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:49.106562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39561 (local address 127.0.0.1:47284)
0807 04:22:49.107173 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:49.107193 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:49.107352 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:49.107740 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:49.107746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:49.107759 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:22:49.108118 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:49.108129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.109136 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.109144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:49.110890 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:49.110907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.111660 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.111669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:49.111866 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:49.111937 (+ 71us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:22:49.114251 (+ 2314us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37707/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 04:22:49.114484 (+ 233us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:39561: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37707/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":323,"threads_started":1}
W20260807 04:22:49.115064 7567 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47284: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37707/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260807 04:22:49.116534 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7496
I20260807 04:22:49.117570 7559 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 04:22:49.129143 7594 webserver.cc:533] Webserver: sslize failed:
E20260807 04:22:49.129269 7594 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 04:22:49.216918 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7496
2026-08-07T04:22:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (736 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T04:22:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:49Z Disabled control of system clock
I20260807 04:22:49.305729 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42229
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:37585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42229
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/testchainca.cert with env {}
W20260807 04:22:49.625960 7599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:49.626262 7599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:49.626349 7599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:49.633433 7599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:49.633538 7599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:49.633582 7599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:49.633641 7599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:49.633703 7599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 04:22:49.633795 7599 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 04:22:49.633836 7599 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 04:22:49.643224 7599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:49.644446 7599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:49.645643 7599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:49.657403 7605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:49.657444 7607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:49.657491 7604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:49.661301 7599 server_base.cc:1034] running on GCE node
I20260807 04:22:49.661988 7599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:49.663257 7599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:49.664480 7599 hybrid_clock.cc:648] HybridClock initialized: now 1786076569664443 us; error 48 us; skew 500 ppm
I20260807 04:22:49.664829 7599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:22:49.667130 7599 webserver.cc:460] Webserver started at http://127.1.244.254:34295/ using document root <none> and password file <none>
I20260807 04:22:49.667826 7599 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:49.668013 7599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:49.668355 7599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:49.671120 7599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "acd5efeb5f174269a6551b5175fd662c"
format_stamp: "Formatted at 2026-08-07 04:22:49 on dist-test-slave-zkpd"
server_key: "64a4cd6133daa7a9cb1d2052b356ed76"
server_key_iv: "e4c646e86f8e849b98754a6a5169e32b"
server_key_version: "encryptionkey@0"
I20260807 04:22:49.671980 7599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "acd5efeb5f174269a6551b5175fd662c"
format_stamp: "Formatted at 2026-08-07 04:22:49 on dist-test-slave-zkpd"
server_key: "64a4cd6133daa7a9cb1d2052b356ed76"
server_key_iv: "e4c646e86f8e849b98754a6a5169e32b"
server_key_version: "encryptionkey@0"
I20260807 04:22:49.677161 7599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:22:49.680546 7613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:49.681842 7599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:22:49.682046 7599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "acd5efeb5f174269a6551b5175fd662c"
format_stamp: "Formatted at 2026-08-07 04:22:49 on dist-test-slave-zkpd"
server_key: "64a4cd6133daa7a9cb1d2052b356ed76"
server_key_iv: "e4c646e86f8e849b98754a6a5169e32b"
server_key_version: "encryptionkey@0"
I20260807 04:22:49.682265 7599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:49.694419 7599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:49.695169 7599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:49.695451 7599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:49.713729 7599 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42229
I20260807 04:22:49.713748 7664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42229 every 8 connection(s)
I20260807 04:22:49.715026 7599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:49.718648 7665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:49.723402 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7599
I20260807 04:22:49.723604 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:49.724366 2003 external_mini_cluster.cc:1428] Setting key 4e8ee74b19f08d83e1370a78997cc75c
I20260807 04:22:49.730304 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c: Bootstrap starting.
I20260807 04:22:49.734162 7665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:49.735299 7665 log.cc:824] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:49.738025 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c: No bootstrap required, opened a new log
I20260807 04:22:49.739913 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:49.726281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.1.244.254:42229)
0807 04:22:49.727441 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:49.727456 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:49.727506 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:22:49.729897 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:49.729927 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:49.730490 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:49.730915 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:49.731470 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.733521 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.734827 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.735974 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.736182 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:49.737317 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:49.737325 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:49.737337 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:22:49.737840 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:49.737913 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:22:49.737934 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:49.738063 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:49.738333 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 04:22:49.738680 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":397,"threads_started":1}
I20260807 04:22:49.743590 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:22:49.746644 7665 raft_consensus.cc:348] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd5efeb5f174269a6551b5175fd662c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42229 } }
I20260807 04:22:49.746994 7665 raft_consensus.cc:374] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:49.747107 7665 raft_consensus.cc:729] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd5efeb5f174269a6551b5175fd662c, State: Initialized, Role: FOLLOWER
I20260807 04:22:49.747680 7665 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd5efeb5f174269a6551b5175fd662c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42229 } }
I20260807 04:22:49.747948 7665 raft_consensus.cc:388] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:49.748087 7665 raft_consensus.cc:482] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:49.748267 7665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:49.750505 7665 raft_consensus.cc:504] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd5efeb5f174269a6551b5175fd662c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42229 } }
I20260807 04:22:49.750970 7665 leader_election.cc:302] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acd5efeb5f174269a6551b5175fd662c; no voters:
I20260807 04:22:49.751508 7665 leader_election.cc:288] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:49.751652 7670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:49.753181 7670 raft_consensus.cc:686] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [term 1 LEADER]: Becoming Leader. State: Replica: acd5efeb5f174269a6551b5175fd662c, State: Running, Role: LEADER
I20260807 04:22:49.753676 7665 sys_catalog.cc:564] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:49.753647 7670 consensus_queue.cc:237] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd5efeb5f174269a6551b5175fd662c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42229 } }
I20260807 04:22:49.756749 7672 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [sys.catalog]: SysCatalogTable state changed. Reason: New leader acd5efeb5f174269a6551b5175fd662c. Latest consensus state: current_term: 1 leader_uuid: "acd5efeb5f174269a6551b5175fd662c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd5efeb5f174269a6551b5175fd662c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42229 } } }
I20260807 04:22:49.757222 7672 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:49.757864 7671 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acd5efeb5f174269a6551b5175fd662c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd5efeb5f174269a6551b5175fd662c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42229 } } }
I20260807 04:22:49.758245 7671 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:49.758634 7679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:49.767274 7679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:49.775846 7679 catalog_manager.cc:1347] Generated new cluster ID: 7d8e88078c6944cda5deb80f0d7587e5
I20260807 04:22:49.775981 7679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:49.785792 7679 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:49.787590 7679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:49.795481 7679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acd5efeb5f174269a6551b5175fd662c: Generated new TSK 0
I20260807 04:22:49.796437 7679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:22:50.265816 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:49.809373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.1.244.254:42229)
0807 04:22:49.809570 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:49.809576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:49.809901 (+ 325us) server_negotiation.cc:407] Connection header received
0807 04:22:49.810108 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:49.810114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:49.810207 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:49.810361 (+ 154us) server_negotiation.cc:226] Negotiated authn=JWT
0807 04:22:49.810802 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.811869 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.813227 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.813959 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.814077 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:49.814516 (+ 439us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 04:22:50.265223 (+450707us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 04:22:50.265585 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49294: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 04:22:50.265918 7668 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49294: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 04:22:50.266149 7695 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:49.809225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42229 (local address 127.0.0.1:49294)
0807 04:22:49.809803 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:49.809819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:49.809937 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:49.810386 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:49.810389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:49.810399 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 04:22:49.810661 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:49.810669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.812035 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.812039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:49.813090 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:49.813100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:49.814122 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:49.814127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:49.814241 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:49.814270 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 04:22:50.265507 (+451237us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 04:22:50.265764 (+ 257us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.244.254:42229: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":309,"threads_started":1}
I20260807 04:22:50.267297 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7599
I20260807 04:22:50.268467 7660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:50.371578 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7599
2026-08-07T04:22:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1136 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (410 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (408 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:51 dist-test-slave-zkpd krb5kdc[7710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:51 dist-test-slave-zkpd krb5kdc[7710](info): set up 2 sockets
Aug 07 04:22:51 dist-test-slave-zkpd krb5kdc[7710](info): commencing operation
krb5kdc: starting...
W20260807 04:22:52.307554 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:52 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:22:52.524796 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39471
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:52.847436 7723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:52.847699 7723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:52.847851 7723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:52.854977 7723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:52.855067 7723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:52.855111 7723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:52.855180 7723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:52.855244 7723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:52.864634 7723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:52.865778 7723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:52.867000 7723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:52.879977 7728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:52.880256 7729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:52.880581 7731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:52.881155 7723 server_base.cc:1034] running on GCE node
I20260807 04:22:52.881922 7723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:52.883294 7723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:52.884505 7723 hybrid_clock.cc:648] HybridClock initialized: now 1786076572884451 us; error 40 us; skew 500 ppm
I20260807 04:22:52.884824 7723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:52 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076572, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:52.887785 7723 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:22:52.889142 7723 webserver.cc:460] Webserver started at http://127.1.244.254:33299/ using document root <none> and password file <none>
I20260807 04:22:52.889768 7723 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:52.889919 7723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:52.890239 7723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:52.893046 7723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "15db4f602b9b4e559cfb7458bfb0cbb7"
format_stamp: "Formatted at 2026-08-07 04:22:52 on dist-test-slave-zkpd"
server_key: "78bd1590ec44bbacecb5e146ff509455"
server_key_iv: "3a02444e1787a9a27b307ad350a84499"
server_key_version: "encryptionkey@0"
I20260807 04:22:52.893808 7723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "15db4f602b9b4e559cfb7458bfb0cbb7"
format_stamp: "Formatted at 2026-08-07 04:22:52 on dist-test-slave-zkpd"
server_key: "78bd1590ec44bbacecb5e146ff509455"
server_key_iv: "3a02444e1787a9a27b307ad350a84499"
server_key_version: "encryptionkey@0"
I20260807 04:22:52.898548 7723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:22:52.901731 7738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:52.902952 7723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 04:22:52.903072 7723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "15db4f602b9b4e559cfb7458bfb0cbb7"
format_stamp: "Formatted at 2026-08-07 04:22:52 on dist-test-slave-zkpd"
server_key: "78bd1590ec44bbacecb5e146ff509455"
server_key_iv: "3a02444e1787a9a27b307ad350a84499"
server_key_version: "encryptionkey@0"
I20260807 04:22:52.903218 7723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:52.916326 7723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:52.919860 7723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:52.920150 7723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:52.938122 7723 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39471
I20260807 04:22:52.938164 7789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39471 every 8 connection(s)
I20260807 04:22:52.939373 7723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:52.942786 7790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:52.944550 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7723
I20260807 04:22:52.944749 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:52.945361 2003 external_mini_cluster.cc:1428] Setting key 52973fbac66e9186c69fcb6cd57abe7f
I20260807 04:22:52.955595 7790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7: Bootstrap starting.
I20260807 04:22:52.959007 7790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:52.960045 7790 log.cc:824] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:22:52 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:52.963356 7790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7: No bootstrap required, opened a new log
I20260807 04:22:52.969977 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:52.947281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.1.244.254:39471)
0807 04:22:52.948156 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:52.948171 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:52.948231 (+ 60us) server_negotiation.cc:407] Connection header received
0807 04:22:52.950644 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:52.950671 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:52.951150 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:52.951573 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:52.952932 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:52.954929 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:52.956257 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:52.957313 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:52.957513 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:52.961631 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:52.961642 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:52.961661 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:52.961701 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:52.965605 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:52.966238 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:52.966249 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:52.966262 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:52.966464 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:52.966800 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:52.966805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:52.966809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:52.967542 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:52.967864 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:52.968189 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 04:22:52.968597 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":329,"threads_started":1}
I20260807 04:22:52.973145 7790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39471 } }
I20260807 04:22:52.973477 7790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:52.973560 7790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15db4f602b9b4e559cfb7458bfb0cbb7, State: Initialized, Role: FOLLOWER
I20260807 04:22:52.974083 7790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39471 } }
I20260807 04:22:52.974334 7790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:52.974440 7790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:52.974560 7790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:52.976739 7790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39471 } }
I20260807 04:22:52.977176 7790 leader_election.cc:302] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15db4f602b9b4e559cfb7458bfb0cbb7; no voters:
I20260807 04:22:52.977676 7790 leader_election.cc:288] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:52.977842 7795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:52.978492 7795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [term 1 LEADER]: Becoming Leader. State: Replica: 15db4f602b9b4e559cfb7458bfb0cbb7, State: Running, Role: LEADER
I20260807 04:22:52.979036 7795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39471 } }
I20260807 04:22:52.979754 7790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:52.981769 7797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15db4f602b9b4e559cfb7458bfb0cbb7. Latest consensus state: current_term: 1 leader_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39471 } } }
I20260807 04:22:52.982229 7797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:52.982553 7796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15db4f602b9b4e559cfb7458bfb0cbb7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39471 } } }
I20260807 04:22:52.982869 7796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:52.984764 7801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:52.989377 7801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:52.997556 7801 catalog_manager.cc:1347] Generated new cluster ID: de614bd344a649cdb6dc2559a556edc3
I20260807 04:22:52.997668 7801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:53.007584 7801 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:53.009521 7801 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:53.017568 7801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15db4f602b9b4e559cfb7458bfb0cbb7: Generated new TSK 0
I20260807 04:22:53.018355 7801 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:22:53.079319 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39471
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:53.408514 7816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:53.408772 7816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:53.408872 7816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:53.415910 7816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:53.416029 7816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:53.416136 7816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:53.425421 7816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:53.426534 7816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:53.427726 7816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:53.440923 7822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:53.440845 7821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:53.440934 7824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:53.441883 7816 server_base.cc:1034] running on GCE node
I20260807 04:22:53.442538 7816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:53.443403 7816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:53.444624 7816 hybrid_clock.cc:648] HybridClock initialized: now 1786076573444590 us; error 46 us; skew 500 ppm
I20260807 04:22:53.444934 7816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:53 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:53.448009 7816 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:22:53.449554 7816 webserver.cc:460] Webserver started at http://127.1.244.193:38917/ using document root <none> and password file <none>
I20260807 04:22:53.450199 7816 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:53.450380 7816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:53.450698 7816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:53.453456 7816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "569d7195adcb41ec82c0ca898056d4e5"
format_stamp: "Formatted at 2026-08-07 04:22:53 on dist-test-slave-zkpd"
server_key: "29bdab066ede184db664e1de0c6d6a29"
server_key_iv: "d3b6869227b02491b3bb48b6a752fbd8"
server_key_version: "encryptionkey@0"
I20260807 04:22:53.454249 7816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "569d7195adcb41ec82c0ca898056d4e5"
format_stamp: "Formatted at 2026-08-07 04:22:53 on dist-test-slave-zkpd"
server_key: "29bdab066ede184db664e1de0c6d6a29"
server_key_iv: "d3b6869227b02491b3bb48b6a752fbd8"
server_key_version: "encryptionkey@0"
I20260807 04:22:53.459681 7816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:22:53.464588 7831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:53.466358 7816 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:22:53.466545 7816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "569d7195adcb41ec82c0ca898056d4e5"
format_stamp: "Formatted at 2026-08-07 04:22:53 on dist-test-slave-zkpd"
server_key: "29bdab066ede184db664e1de0c6d6a29"
server_key_iv: "d3b6869227b02491b3bb48b6a752fbd8"
server_key_version: "encryptionkey@0"
I20260807 04:22:53.466746 7816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:53.498756 7816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:53.502657 7816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:53.502962 7816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:53.503952 7816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:53.505263 7816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:53.505368 7816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:53.505535 7816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:53.505607 7816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:53.538787 7816 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:41287
I20260807 04:22:53.538829 7944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:41287 every 8 connection(s)
I20260807 04:22:53.540194 7816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:53.550336 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7816
I20260807 04:22:53.550555 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:53.551429 2003 external_mini_cluster.cc:1428] Setting key 0397812c44f432679c4ecbf426474003
Aug 07 04:22:53 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:53.561084 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:53.542705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:34957 (local address 127.1.244.254:39471)
0807 04:22:53.543165 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:53.543169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:53.544089 (+ 920us) server_negotiation.cc:407] Connection header received
0807 04:22:53.545920 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:53.545926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:53.546026 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:53.546151 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:53.547634 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:53.548451 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:53.550169 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:53.550695 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:53.550819 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:53.555187 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:53.555192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:53.555195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:53.555232 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:53.557875 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:53.558782 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:53.558785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:53.558788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:53.558903 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:53.559381 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:53.559384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:53.559386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:53.559630 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:53.559737 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:53.560640 (+ 903us) server_negotiation.cc:299] Negotiation successful
0807 04:22:53.560805 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":260,"threads_started":1}
I20260807 04:22:53.562573 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:53.543156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39471 (local address 127.1.244.193:34957)
0807 04:22:53.543842 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:53.543960 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:53.545630 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:53.546400 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:53.546413 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:53.546890 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:53.547462 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:53.547480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:53.548617 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:53.548621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:53.549997 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:53.550014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:53.550939 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:53.550948 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:53.551250 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:53.552170 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:53.552215 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:53.554981 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:53.558020 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:53.558032 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:53.558051 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:53.558668 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:53.559012 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:53.559019 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:53.559025 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:53.559272 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:53.559755 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:53.559765 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:53.560254 (+ 489us) client_negotiation.cc:769] Sending connection context
0807 04:22:53.560566 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 04:22:53.561133 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":290,"threads_started":1}
I20260807 04:22:53.564127 7945 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39471
I20260807 04:22:53.564440 7945 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:53.565145 7945 heartbeater.cc:511] Master 127.1.244.254:39471 requested a full tablet report, sending...
I20260807 04:22:53.567226 7755 ts_manager.cc:194] Registered new tserver with Master: 569d7195adcb41ec82c0ca898056d4e5 (127.1.244.193:41287)
I20260807 04:22:53.568732 7755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:34957
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:22:53.602839 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39471
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:53.924064 7951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:53.924444 7951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:53.924620 7951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:53.931753 7951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:53.931902 7951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:53.932093 7951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:53.942233 7951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:53.943394 7951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:53.944725 7951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:53.958041 7956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:53.958003 7959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:53.958024 7957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:53.958853 7951 server_base.cc:1034] running on GCE node
I20260807 04:22:53.959450 7951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:53.960392 7951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:53.961619 7951 hybrid_clock.cc:648] HybridClock initialized: now 1786076573961573 us; error 52 us; skew 500 ppm
I20260807 04:22:53.961922 7951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:53 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:53.965142 7951 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:22:53.966480 7951 webserver.cc:460] Webserver started at http://127.1.244.194:43755/ using document root <none> and password file <none>
I20260807 04:22:53.967130 7951 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:53.967280 7951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:53.967592 7951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:53.970611 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "f33ee762bc764bcba7e4d095fcedde60"
format_stamp: "Formatted at 2026-08-07 04:22:53 on dist-test-slave-zkpd"
server_key: "5bff57ae080d6c3f2db89d41135d41f4"
server_key_iv: "03eda146a0e51b2b070ad3291679faf3"
server_key_version: "encryptionkey@0"
I20260807 04:22:53.971487 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "f33ee762bc764bcba7e4d095fcedde60"
format_stamp: "Formatted at 2026-08-07 04:22:53 on dist-test-slave-zkpd"
server_key: "5bff57ae080d6c3f2db89d41135d41f4"
server_key_iv: "03eda146a0e51b2b070ad3291679faf3"
server_key_version: "encryptionkey@0"
I20260807 04:22:53.976550 7951 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 04:22:53.979753 7966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:53.981199 7951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:22:53.981372 7951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "f33ee762bc764bcba7e4d095fcedde60"
format_stamp: "Formatted at 2026-08-07 04:22:53 on dist-test-slave-zkpd"
server_key: "5bff57ae080d6c3f2db89d41135d41f4"
server_key_iv: "03eda146a0e51b2b070ad3291679faf3"
server_key_version: "encryptionkey@0"
I20260807 04:22:53.981578 7951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:53.998598 7951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:54.002878 7951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:54.003168 7951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:54.004133 7951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:54.005389 7951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:54.005494 7951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:54.005635 7951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:54.005702 7951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:54.037922 7951 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:35607
I20260807 04:22:54.037976 8080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:35607 every 8 connection(s)
I20260807 04:22:54.039206 7951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:54.042603 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 7951
I20260807 04:22:54.042814 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:54.043380 2003 external_mini_cluster.cc:1428] Setting key 71d57d84222746150792b76b39776bde
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:54.059508 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.041709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41491 (local address 127.1.244.254:39471)
0807 04:22:54.041956 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.041964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.043208 (+ 1244us) server_negotiation.cc:407] Connection header received
0807 04:22:54.045198 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.045204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.045294 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.045418 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.046909 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.047626 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.049216 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.049641 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.049807 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.053528 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.053533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.053537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.053569 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.056316 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.057247 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.057251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.057253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.057382 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.057853 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.057856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.057858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.058116 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.058224 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.059094 (+ 870us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.059265 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 04:22:54.061169 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.042306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39471 (local address 127.1.244.194:41491)
0807 04:22:54.043063 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:54.043099 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:54.044924 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:54.045642 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:54.045650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:54.046187 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:54.046748 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:54.046765 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.047781 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.047787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:54.049069 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:54.049081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.049883 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.049891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:54.050121 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.050780 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:54.050805 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:54.053297 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:54.056453 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:54.056466 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:54.056484 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:54.057128 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:54.057495 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:54.057502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:54.057509 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:54.057744 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:54.058248 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:54.058258 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:54.058709 (+ 451us) client_negotiation.cc:769] Sending connection context
0807 04:22:54.059014 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 04:22:54.059581 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":328,"threads_started":1}
I20260807 04:22:54.062633 8081 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39471
I20260807 04:22:54.062968 8081 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:54.063911 8081 heartbeater.cc:511] Master 127.1.244.254:39471 requested a full tablet report, sending...
I20260807 04:22:54.065557 7755 ts_manager.cc:194] Registered new tserver with Master: f33ee762bc764bcba7e4d095fcedde60 (127.1.244.194:35607)
I20260807 04:22:54.066327 7755 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41491
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:22:54.094916 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39471
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:40075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:54.415172 8086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:54.415458 8086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:54.415648 8086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:54.422845 8086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:54.422979 8086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:54.423168 8086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:54.432819 8086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:40075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:54.434032 8086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:54.435314 8086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:54.448105 8091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:54.448235 8094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:54.448143 8092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:54.448846 8086 server_base.cc:1034] running on GCE node
I20260807 04:22:54.449451 8086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:54.450302 8086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:54.451542 8086 hybrid_clock.cc:648] HybridClock initialized: now 1786076574451487 us; error 50 us; skew 500 ppm
I20260807 04:22:54.451834 8086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.454859 8086 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:22:54.456312 8086 webserver.cc:460] Webserver started at http://127.1.244.195:37893/ using document root <none> and password file <none>
I20260807 04:22:54.456940 8086 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:54.457085 8086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:54.457425 8086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:54.460315 8086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "175a2354977e4aa5bce50eb6cedc688a"
format_stamp: "Formatted at 2026-08-07 04:22:54 on dist-test-slave-zkpd"
server_key: "0ffa9a9b0a69a40d51d6538fd4adf3d9"
server_key_iv: "6904d19e29d7c4f88a2e5a1db688d319"
server_key_version: "encryptionkey@0"
I20260807 04:22:54.461087 8086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "175a2354977e4aa5bce50eb6cedc688a"
format_stamp: "Formatted at 2026-08-07 04:22:54 on dist-test-slave-zkpd"
server_key: "0ffa9a9b0a69a40d51d6538fd4adf3d9"
server_key_iv: "6904d19e29d7c4f88a2e5a1db688d319"
server_key_version: "encryptionkey@0"
I20260807 04:22:54.466051 8086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 04:22:54.469252 8101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:54.470469 8086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 04:22:54.470624 8086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "175a2354977e4aa5bce50eb6cedc688a"
format_stamp: "Formatted at 2026-08-07 04:22:54 on dist-test-slave-zkpd"
server_key: "0ffa9a9b0a69a40d51d6538fd4adf3d9"
server_key_iv: "6904d19e29d7c4f88a2e5a1db688d319"
server_key_version: "encryptionkey@0"
I20260807 04:22:54.470866 8086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:54.498996 8086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:54.502746 8086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:54.503021 8086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:54.503937 8086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:54.505245 8086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:54.505347 8086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:54.505486 8086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:54.505554 8086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:54.538367 8086 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:44353
I20260807 04:22:54.538386 8215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:44353 every 8 connection(s)
I20260807 04:22:54.539650 8086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:54.545938 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8086
I20260807 04:22:54.546133 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:54.546566 2003 external_mini_cluster.cc:1428] Setting key 25d0b0b120438e277bfc79a5fe87d9f3
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:22:54.559382 7947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.542582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:38861 (local address 127.1.244.254:39471)
0807 04:22:54.542759 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.542764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.543996 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 04:22:54.545810 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.545816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.545940 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.546050 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.547667 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.548533 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.550070 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.550478 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.550612 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.554040 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.554050 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.554053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.554084 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.556477 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.557156 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.557163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.557167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.557301 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.557712 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.557720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.557723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.557984 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.558144 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.558771 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.558988 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:22:54.560456 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.543097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39471 (local address 127.1.244.195:38861)
0807 04:22:54.543818 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:54.543859 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:54.545533 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:54.546287 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:54.546295 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:54.546843 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:54.547486 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:54.547509 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.548678 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.548684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:54.549920 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:54.549933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.550696 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.550705 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:54.550940 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.551533 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:54.551555 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:54.553834 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:54.556615 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:54.556624 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:54.556637 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:54.557036 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:54.557414 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:54.557421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:54.557425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:54.557602 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:54.558117 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:54.558124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:54.558420 (+ 296us) client_negotiation.cc:769] Sending connection context
0807 04:22:54.558664 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 04:22:54.559198 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":316,"threads_started":1}
I20260807 04:22:54.561923 8216 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39471
I20260807 04:22:54.562201 8216 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:54.562861 8216 heartbeater.cc:511] Master 127.1.244.254:39471 requested a full tablet report, sending...
I20260807 04:22:54.564214 7754 ts_manager.cc:194] Registered new tserver with Master: 175a2354977e4aa5bce50eb6cedc688a (127.1.244.195:44353)
I20260807 04:22:54.565032 7754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:38861
I20260807 04:22:54.571614 7945 heartbeater.cc:503] Master 127.1.244.254:39471 was elected leader, sending a full tablet report...
I20260807 04:22:54.574144 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/test-admin.keytab.
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.604725 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 04:22:54.609758 8223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:54.607340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.1.244.193:41287)
0807 04:22:54.608068 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.608076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.608104 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:22:54.608167 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.608173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.608390 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.608528 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.609355 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46058: BlockingRecv error: recv got EOF from 127.0.0.1:46058 (error 108)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":362,"threads_started":1}
W20260807 04:22:54.609810 2003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc' of type FILE
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.611339 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.612702 2003 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:54.627688 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.614144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.1.244.193:41287)
0807 04:22:54.614343 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.614349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.614588 (+ 239us) server_negotiation.cc:407] Connection header received
0807 04:22:54.614780 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.614785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.614855 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.614964 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.615954 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.616900 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.618085 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.618646 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.618785 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.622306 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.622317 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.622330 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.622360 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.624999 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.625653 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.625660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.625663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.625781 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.626153 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.626158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.626161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.626511 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.626807 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.627143 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.627413 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 04:22:54.633307 8223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:54.631528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46080 (local address 127.1.244.193:41287)
0807 04:22:54.631753 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.631758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.632048 (+ 290us) server_negotiation.cc:407] Connection header received
0807 04:22:54.632177 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.632180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.632234 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.632309 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.633070 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46080: BlockingRecv error: recv got EOF from 127.0.0.1:46080 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 04:22:54.633476 2003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc' of type FILE
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.634883 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.636123 2003 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:54.649271 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.637428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.1.244.193:41287)
0807 04:22:54.637594 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.637599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.637738 (+ 139us) server_negotiation.cc:407] Connection header received
0807 04:22:54.637928 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.637933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.637996 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.638089 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.638953 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.639668 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.640913 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.641317 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.641441 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.644707 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.644712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.644715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.644742 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.647176 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.647681 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.647684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.647686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.647796 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.648174 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.648177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.648179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.648410 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.648591 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.648829 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.649033 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 04:22:54.652974 8223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:22:54.651496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.1.244.193:41287)
0807 04:22:54.651679 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.651684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.651844 (+ 160us) server_negotiation.cc:407] Connection header received
0807 04:22:54.652010 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.652015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.652086 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.652159 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.652757 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46104: BlockingRecv error: recv got EOF from 127.0.0.1:46104 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 04:22:54.653123 2003 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786076526463677-2003-0/krb5kdc/krb5cc' of type FILE
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.654539 2003 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:54.655718 2003 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:22:54.668344 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.657021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.1.244.193:41287)
0807 04:22:54.657218 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.657222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.657300 (+ 78us) server_negotiation.cc:407] Connection header received
0807 04:22:54.657398 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.657402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.657458 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.657528 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.658378 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.659032 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.660215 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.660608 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.660695 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.663696 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.663700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.663703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.663728 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.666227 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.666771 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.666775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.666777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.666916 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.667211 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.667214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.667216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.667438 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.667661 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.667800 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.668061 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:22:54.669770 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7816
I20260807 04:22:54.682327 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.670435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.1.244.193:41287)
0807 04:22:54.670556 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.670560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.671739 (+ 1179us) server_negotiation.cc:407] Connection header received
0807 04:22:54.671960 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.671964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.672013 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.672108 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.672953 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.673585 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.674953 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.675423 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.675547 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.677606 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.677610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.677613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.677639 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.679874 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.680502 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.680509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.680512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.680645 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.681038 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.681043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.681046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.681279 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.681537 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.681845 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.682039 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 04:22:54.682864 7940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:54.766877 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7816
I20260807 04:22:54.785144 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7951
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:22:54.800436 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.786155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.1.244.194:35607)
0807 04:22:54.786717 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.786724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.786748 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:22:54.786809 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.786816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.787023 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.787154 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.788159 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.789130 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.790291 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.790793 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.790881 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.794832 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.794843 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.794856 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.794889 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.797643 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.798325 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.798332 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.798335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.798467 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.798870 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.798874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.798876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.799213 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.799549 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.799842 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.800125 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":290,"threads_started":1}
I20260807 04:22:54.801652 8075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:54.882654 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7951
I20260807 04:22:54.900996 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8086
Aug 07 04:22:54 dist-test-slave-zkpd krb5kdc[7710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:22:54.916570 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:54.901967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.1.244.195:44353)
0807 04:22:54.902630 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:54.902637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:54.902664 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:22:54.902722 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:54.902727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:54.902935 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:54.903100 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:54.904190 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.905212 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.906435 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:54.906979 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:54.907118 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:54.911095 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:54.911106 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:54.911119 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:54.911151 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:54.913825 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.914471 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.914477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.914480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.914598 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:54.914995 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:54.914999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:54.915001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:54.915314 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:54.915644 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:54.915932 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 04:22:54.916237 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":324,"threads_started":1}
I20260807 04:22:54.917810 8211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:55.000034 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8086
I20260807 04:22:55.017338 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 7723
I20260807 04:22:55.018512 7785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:55.120105 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 7723
2026-08-07T04:22:55Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3933 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:22:55 dist-test-slave-zkpd krb5kdc[8243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:22:55 dist-test-slave-zkpd krb5kdc[8243](info): set up 2 sockets
Aug 07 04:22:55 dist-test-slave-zkpd krb5kdc[8243](info): commencing operation
krb5kdc: starting...
W20260807 04:22:56.235777 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:22:56 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:22:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:22:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
I20260807 04:22:56.459196 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39343
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:56.780191 8257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:56.780508 8257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:56.780676 8257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:56.787773 8257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:56.787927 8257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:56.788043 8257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:56.788146 8257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:56.788221 8257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:56.797814 8257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:56.799050 8257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:56.800371 8257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:56.812012 8265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:56.812165 8263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:56.812132 8262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:56.813060 8257 server_base.cc:1034] running on GCE node
I20260807 04:22:56.813832 8257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:56.815128 8257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:56.816344 8257 hybrid_clock.cc:648] HybridClock initialized: now 1786076576816306 us; error 47 us; skew 500 ppm
I20260807 04:22:56.816695 8257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:56 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:56.823650 8257 init.cc:377] Logged in from keytab as oryx/127.1.244.254@KRBTEST.COM (short username oryx)
I20260807 04:22:56.825187 8257 webserver.cc:460] Webserver started at http://127.1.244.254:36019/ using document root <none> and password file <none>
I20260807 04:22:56.825860 8257 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:56.826004 8257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:56.826301 8257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:56.829196 8257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "0f454652f335495597b168618f546d2a"
format_stamp: "Formatted at 2026-08-07 04:22:56 on dist-test-slave-zkpd"
server_key: "b9dc03d4a3b65bdfc1461ccb8af77b59"
server_key_iv: "2fdb1e76e556c02facef3dbaa645bcaa"
server_key_version: "encryptionkey@0"
I20260807 04:22:56.829974 8257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "0f454652f335495597b168618f546d2a"
format_stamp: "Formatted at 2026-08-07 04:22:56 on dist-test-slave-zkpd"
server_key: "b9dc03d4a3b65bdfc1461ccb8af77b59"
server_key_iv: "2fdb1e76e556c02facef3dbaa645bcaa"
server_key_version: "encryptionkey@0"
I20260807 04:22:56.834861 8257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 04:22:56.838066 8272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:56.839267 8257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 04:22:56.839454 8257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "0f454652f335495597b168618f546d2a"
format_stamp: "Formatted at 2026-08-07 04:22:56 on dist-test-slave-zkpd"
server_key: "b9dc03d4a3b65bdfc1461ccb8af77b59"
server_key_iv: "2fdb1e76e556c02facef3dbaa645bcaa"
server_key_version: "encryptionkey@0"
I20260807 04:22:56.839638 8257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:56.856678 8257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:56.860213 8257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:56.860491 8257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:56.878109 8257 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39343
I20260807 04:22:56.878152 8324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39343 every 8 connection(s)
I20260807 04:22:56.879382 8257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:22:56.880436 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8257
I20260807 04:22:56.880673 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:22:56.881563 2003 external_mini_cluster.cc:1428] Setting key 93f629fe899c71f5eb6c36e1a0dd5173
I20260807 04:22:56.884233 8325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:22:56 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:22:56.898277 8325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Bootstrap starting.
I20260807 04:22:56.901604 8325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:56.902622 8325 log.cc:824] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:56.905877 8325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: No bootstrap required, opened a new log
I20260807 04:22:56.906364 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:56.883602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.1.244.254:39343)
0807 04:22:56.884510 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:56.884525 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:56.884573 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:22:56.886851 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:56.886878 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:56.887346 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:56.887790 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:56.889666 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:56.892007 (+ 2341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:56.893778 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:56.894571 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:56.894772 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:56.899030 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:56.899038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:56.899051 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:56.899084 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:56.902213 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:56.902873 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:56.902880 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:56.902887 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:56.903082 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:56.903543 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:56.903546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:56.903548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:56.904146 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:56.904426 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:56.904739 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 04:22:56.905138 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":396,"threads_started":1}
I20260807 04:22:56.913759 8325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } }
I20260807 04:22:56.914072 8325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:56.914144 8325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f454652f335495597b168618f546d2a, State: Initialized, Role: FOLLOWER
I20260807 04:22:56.914565 8325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } }
I20260807 04:22:56.914757 8325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:22:56.914846 8325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:22:56.914978 8325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:56.916975 8325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } }
I20260807 04:22:56.917385 8325 leader_election.cc:302] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f454652f335495597b168618f546d2a; no voters:
I20260807 04:22:56.918534 8331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:22:56.918684 8325 leader_election.cc:288] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:22:56.919565 8331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 LEADER]: Becoming Leader. State: Replica: 0f454652f335495597b168618f546d2a, State: Running, Role: LEADER
I20260807 04:22:56.920185 8331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } }
I20260807 04:22:56.922730 8325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:56.924223 8332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f454652f335495597b168618f546d2a. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } }
I20260807 04:22:56.924572 8332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:56.922910 8333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } }
I20260807 04:22:56.925277 8333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:56.928618 8337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:22:56.934672 8337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:22:56.959177 8337 catalog_manager.cc:1347] Generated new cluster ID: fd18d5a3c3a34e82b9af0a0b9bbc26d1
I20260807 04:22:56.959306 8337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:22:56.990633 8337 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:22:56.992479 8337 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:22:57.008913 8337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Generated new TSK 0
I20260807 04:22:57.009694 8337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
I20260807 04:22:57.071801 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39343
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:57.404395 8353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:57.404666 8353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:57.404863 8353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:57.412338 8353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:57.412462 8353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:57.412648 8353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:22:57.422309 8353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:57.423581 8353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:57.424959 8353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:57.438392 8358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:57.438321 8361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:57.438338 8359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:57.439217 8353 server_base.cc:1034] running on GCE node
I20260807 04:22:57.439870 8353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:57.440852 8353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:57.442099 8353 hybrid_clock.cc:648] HybridClock initialized: now 1786076577442061 us; error 46 us; skew 500 ppm
I20260807 04:22:57.442418 8353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:57 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076577, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:57.445613 8353 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260807 04:22:57.447036 8353 webserver.cc:460] Webserver started at http://127.1.244.193:34073/ using document root <none> and password file <none>
I20260807 04:22:57.447721 8353 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:57.447865 8353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:57.448222 8353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:57.451333 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "c676684f55c341c683428b058cb45faf"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "d0e20fd9042c1e9526dac6feff3babb6"
server_key_iv: "ce823327449b61a6762703561901e879"
server_key_version: "encryptionkey@0"
I20260807 04:22:57.452188 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "c676684f55c341c683428b058cb45faf"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "d0e20fd9042c1e9526dac6feff3babb6"
server_key_iv: "ce823327449b61a6762703561901e879"
server_key_version: "encryptionkey@0"
I20260807 04:22:57.457206 8353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:22:57.462188 8368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:57.464054 8353 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260807 04:22:57.464282 8353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "c676684f55c341c683428b058cb45faf"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "d0e20fd9042c1e9526dac6feff3babb6"
server_key_iv: "ce823327449b61a6762703561901e879"
server_key_version: "encryptionkey@0"
I20260807 04:22:57.464541 8353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:57.483466 8353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:57.487299 8353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:57.487579 8353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:57.488569 8353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:57.489887 8353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:57.490005 8353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:57.490157 8353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:57.490236 8353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:57.523200 8353 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44963
I20260807 04:22:57.523247 8482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44963 every 8 connection(s)
I20260807 04:22:57.524569 8353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:22:57.533898 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8353
I20260807 04:22:57.534116 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:22:57.534911 2003 external_mini_cluster.cc:1428] Setting key fac825f32e0634bf0cf0ecd4d511819c
Aug 07 04:22:57 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076577, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:22:57.546247 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:57.527047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:55083 (local address 127.1.244.254:39343)
0807 04:22:57.527535 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:57.527542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:57.528488 (+ 946us) server_negotiation.cc:407] Connection header received
0807 04:22:57.530306 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:57.530310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:57.530374 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:57.530474 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:57.531955 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:57.532645 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:57.534446 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:57.535085 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:57.535220 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:57.539148 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:57.539157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:57.539165 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:57.539210 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:57.542789 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:57.543658 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:57.543665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:57.543671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:57.543819 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:57.544335 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:57.544342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:57.544347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:57.544720 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:57.544876 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:57.545519 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 04:22:57.545767 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260807 04:22:57.546873 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:57.527532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:55083)
0807 04:22:57.528324 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:57.528367 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:57.530057 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:57.530724 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:57.530733 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:57.531239 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:57.531776 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:57.531794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:57.532941 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:57.532945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:57.534289 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:57.534302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:57.535291 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:57.535295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:57.535534 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:57.536287 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:57.536323 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:57.538936 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:57.542947 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:57.542953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:57.542982 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:57.543529 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:57.543980 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:57.543984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:57.543986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:57.544219 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:57.544843 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:57.544849 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:57.545149 (+ 300us) client_negotiation.cc:769] Sending connection context
0807 04:22:57.545362 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 04:22:57.545719 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":328,"threads_started":1}
I20260807 04:22:57.548454 8483 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39343
I20260807 04:22:57.548750 8483 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:57.549436 8483 heartbeater.cc:511] Master 127.1.244.254:39343 requested a full tablet report, sending...
I20260807 04:22:57.551676 8290 ts_manager.cc:194] Registered new tserver with Master: c676684f55c341c683428b058cb45faf (127.1.244.193:44963)
I20260807 04:22:57.553929 8290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:55083
WARNING: no policy specified for oryx/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
I20260807 04:22:57.588397 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39343
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:57.918413 8489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:57.918672 8489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:57.918866 8489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:57.926054 8489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:57.926183 8489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:57.926383 8489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:22:57.935644 8489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:57.936820 8489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:57.938094 8489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:57.950759 8495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:57.950836 8494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:57.950814 8497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:57.951828 8489 server_base.cc:1034] running on GCE node
I20260807 04:22:57.952423 8489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:57.953245 8489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:57.954468 8489 hybrid_clock.cc:648] HybridClock initialized: now 1786076577954412 us; error 41 us; skew 500 ppm
I20260807 04:22:57.954772 8489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:57 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076577, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:57.957896 8489 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260807 04:22:57.959331 8489 webserver.cc:460] Webserver started at http://127.1.244.194:42445/ using document root <none> and password file <none>
I20260807 04:22:57.960064 8489 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:57.960208 8489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:57.960510 8489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:57.963291 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "7b8f0386037d4c9aa7e6d041a2de9c57"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "86a7d51476bea0d0346ee05ff0dd3754"
server_key_iv: "9045b94cb02902ba088fadabe63be716"
server_key_version: "encryptionkey@0"
I20260807 04:22:57.964123 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "7b8f0386037d4c9aa7e6d041a2de9c57"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "86a7d51476bea0d0346ee05ff0dd3754"
server_key_iv: "9045b94cb02902ba088fadabe63be716"
server_key_version: "encryptionkey@0"
I20260807 04:22:57.969015 8489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:22:57.972383 8504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:57.973604 8489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:22:57.973793 8489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "7b8f0386037d4c9aa7e6d041a2de9c57"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "86a7d51476bea0d0346ee05ff0dd3754"
server_key_iv: "9045b94cb02902ba088fadabe63be716"
server_key_version: "encryptionkey@0"
I20260807 04:22:57.974002 8489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:57.993252 8489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:57.997418 8489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:57.997745 8489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:57.998654 8489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:57.999976 8489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:58.000079 8489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:58.000236 8489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:58.000310 8489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:58.033324 8489 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42793
I20260807 04:22:58.033409 8618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42793 every 8 connection(s)
I20260807 04:22:58.034569 8489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:22:58.039826 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8489
I20260807 04:22:58.040035 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:22:58.040625 2003 external_mini_cluster.cc:1428] Setting key ac8dff3e5c948afa1e44ca75daf71d7e
Aug 07 04:22:58 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076577, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:22:58.054783 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:58.037061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:51655 (local address 127.1.244.254:39343)
0807 04:22:58.037243 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:58.037248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:58.038363 (+ 1115us) server_negotiation.cc:407] Connection header received
0807 04:22:58.040569 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:58.040573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:58.040641 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:58.040766 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:58.042540 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.043356 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.045026 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.045452 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.045587 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:58.049082 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:58.049087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:58.049090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:58.049119 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:58.051674 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:58.052444 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:58.052451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:58.052459 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:58.052582 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:58.053044 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:58.053048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:58.053053 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:58.053337 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:58.053499 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:58.054197 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 04:22:58.054440 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:22:58.055706 8621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:58.037517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:51655)
0807 04:22:58.038205 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:58.038242 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:58.040258 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:58.041060 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:58.041071 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:58.041596 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:58.042338 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:58.042363 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.043505 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.043511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:58.044877 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:58.044889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.045678 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.045684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:58.045932 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:58.046602 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:58.046628 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:58.048860 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:58.051847 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:58.051868 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:58.051882 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:58.052317 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:58.052695 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:58.052700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:58.052704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:58.052921 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:58.053470 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:58.053478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:58.053821 (+ 343us) client_negotiation.cc:769] Sending connection context
0807 04:22:58.054076 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 04:22:58.054574 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":269,"threads_started":1}
I20260807 04:22:58.057315 8619 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39343
I20260807 04:22:58.057693 8619 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:58.058578 8619 heartbeater.cc:511] Master 127.1.244.254:39343 requested a full tablet report, sending...
I20260807 04:22:58.060250 8290 ts_manager.cc:194] Registered new tserver with Master: 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793)
I20260807 04:22:58.061609 8290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:51655
WARNING: no policy specified for oryx/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
I20260807 04:22:58.093389 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39343
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:58.416837 8624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:58.417102 8624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:58.417196 8624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:58.424414 8624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:58.424517 8624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:58.424657 8624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:22:58.434473 8624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:58.435645 8624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:58.437031 8624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:58.449908 8633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:58.449965 8631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:58.449955 8630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:58.450981 8624 server_base.cc:1034] running on GCE node
I20260807 04:22:58.451529 8624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:58.452436 8624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:58.453660 8624 hybrid_clock.cc:648] HybridClock initialized: now 1786076578453620 us; error 48 us; skew 500 ppm
I20260807 04:22:58.453982 8624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:58 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:58.457140 8624 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260807 04:22:58.458578 8624 webserver.cc:460] Webserver started at http://127.1.244.195:32895/ using document root <none> and password file <none>
I20260807 04:22:58.459182 8624 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:58.459278 8624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:58.459522 8624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:58.462338 8624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "708723687b6f27f05ae12cd5ada3ef66"
server_key_iv: "ea21568ef04d88b3822d2bc80eca185e"
server_key_version: "encryptionkey@0"
I20260807 04:22:58.463061 8624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "708723687b6f27f05ae12cd5ada3ef66"
server_key_iv: "ea21568ef04d88b3822d2bc80eca185e"
server_key_version: "encryptionkey@0"
I20260807 04:22:58.468001 8624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:22:58.471410 8640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:58.472759 8624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:22:58.472924 8624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "708723687b6f27f05ae12cd5ada3ef66"
server_key_iv: "ea21568ef04d88b3822d2bc80eca185e"
server_key_version: "encryptionkey@0"
I20260807 04:22:58.473124 8624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:58.494141 8624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:58.498322 8624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:58.498626 8624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:58.499576 8624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:22:58.500901 8624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:22:58.500979 8624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:58.501144 8624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:22:58.501211 8624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:58.536754 8624 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:35953
I20260807 04:22:58.536804 8755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:35953 every 8 connection(s)
I20260807 04:22:58.537994 8624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:22:58.546099 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8624
I20260807 04:22:58.546276 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:22:58.546836 2003 external_mini_cluster.cc:1428] Setting key 5aad094251450dda70cb06ff8789c54c
Aug 07 04:22:58 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:22:58.556918 8483 heartbeater.cc:503] Master 127.1.244.254:39343 was elected leader, sending a full tablet report...
I20260807 04:22:58.558063 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:58.540458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:51731 (local address 127.1.244.254:39343)
0807 04:22:58.540639 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:58.540643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:58.541710 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 04:22:58.543770 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:58.543774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:58.543842 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:58.543995 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:58.545383 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.546132 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.548263 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.548671 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.548818 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:58.552292 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:58.552297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:58.552300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:58.552332 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:58.554926 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:58.555635 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:58.555642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:58.555645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:58.555788 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:58.556235 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:58.556242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:58.556245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:58.556510 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:58.556672 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:58.557407 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 04:22:58.557677 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:22:58.558923 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:58.540899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:51731)
0807 04:22:58.541551 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:58.541594 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:58.543494 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:58.544197 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:58.544205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:58.544672 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:58.545234 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:58.545252 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.546316 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.546324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:58.548111 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:58.548130 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:58.548909 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:58.548920 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:58.549170 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:58.549841 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:58.549869 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:58.552081 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:58.555069 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:58.555078 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:58.555093 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:58.555515 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:58.555923 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:58.555928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:58.555932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:58.556121 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:58.556659 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:58.556667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:58.556991 (+ 324us) client_negotiation.cc:769] Sending connection context
0807 04:22:58.557300 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 04:22:58.557819 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":253,"threads_started":1}
I20260807 04:22:58.560238 8756 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39343
I20260807 04:22:58.560544 8756 heartbeater.cc:464] Registering TS with master...
I20260807 04:22:58.561345 8756 heartbeater.cc:511] Master 127.1.244.254:39343 requested a full tablet report, sending...
I20260807 04:22:58.562759 8290 ts_manager.cc:194] Registered new tserver with Master: de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953)
I20260807 04:22:58.563513 8290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:51731
I20260807 04:22:58.574116 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.253.keytab.
Entry for principal oryx/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.253.keytab.
I20260807 04:22:58.625160 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:38199
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.253.keytab
--principal=oryx/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39343,127.1.244.253:38199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:58.953908 8761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:58.954170 8761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:58.954357 8761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:58.961444 8761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:58.961539 8761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:58.961584 8761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:58.961663 8761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:58.961750 8761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:58.971532 8761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39343,127.1.244.253:38199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.253.keytab
--principal=oryx/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:38199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
W20260807 04:22:58.971874 8761 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.244.254:39343,127.1.244.253:38199'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 04:22:58.972891 8761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:58.974195 8761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:58.985997 8766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:58.986084 8767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:58.986060 8769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:58.987052 8761 server_base.cc:1034] running on GCE node
I20260807 04:22:58.987794 8761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:58.989071 8761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:58.990307 8761 hybrid_clock.cc:648] HybridClock initialized: now 1786076578990243 us; error 40 us; skew 500 ppm
I20260807 04:22:58.990636 8761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:58 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:58.993669 8761 init.cc:377] Logged in from keytab as oryx/127.1.244.253@KRBTEST.COM (short username oryx)
I20260807 04:22:58.995083 8761 webserver.cc:460] Webserver started at http://127.1.244.253:43251/ using document root <none> and password file <none>
I20260807 04:22:58.995749 8761 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:58.995875 8761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:58.996258 8761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:58.999056 8761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/instance:
uuid: "d47bccfe2ef84e068685777520783522"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "6da712b1c44f6025a7519dbe5a45795e"
server_key_iv: "6958b8781d13ad50e655833d3041b8df"
server_key_version: "encryptionkey@0"
I20260807 04:22:58.999816 8761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal/instance:
uuid: "d47bccfe2ef84e068685777520783522"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "6da712b1c44f6025a7519dbe5a45795e"
server_key_iv: "6958b8781d13ad50e655833d3041b8df"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.004699 8761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 04:22:59.007979 8776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:59.009209 8761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:22:59.009370 8761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
uuid: "d47bccfe2ef84e068685777520783522"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "6da712b1c44f6025a7519dbe5a45795e"
server_key_iv: "6958b8781d13ad50e655833d3041b8df"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.009603 8761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:59.039467 8761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:59.043375 8761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:59.043654 8761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:59.062223 8761 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:38199
I20260807 04:22:59.062245 8827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:38199 every 8 connection(s)
I20260807 04:22:59.063540 8761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
I20260807 04:22:59.064836 8619 heartbeater.cc:503] Master 127.1.244.254:39343 was elected leader, sending a full tablet report...
I20260807 04:22:59.067375 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8761
I20260807 04:22:59.067756 8828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:59.077585 8828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:22:59.091336 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.069688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.1.244.253:38199)
0807 04:22:59.070477 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.070491 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.070528 (+ 37us) server_negotiation.cc:407] Connection header received
0807 04:22:59.072482 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.072503 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.072850 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.073156 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.075057 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.076585 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.078404 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.079131 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.079252 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.084219 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.084230 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.084240 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.084279 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.087427 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.088191 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.088199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.088214 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.088364 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.088781 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.088789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.088794 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.089326 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.089639 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.090091 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.090446 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":365,"threads_started":1}
I20260807 04:22:59.095773 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.080414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:39596)
0807 04:22:59.081105 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.081152 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.081310 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.081986 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.081993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.082697 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.083037 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.083050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.084184 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.084188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.085255 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.085270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.086267 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.086272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.086390 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.086987 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.087003 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.089185 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.093362 (+ 4177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.093371 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.093377 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.093972 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.094301 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.094304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.094307 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.094426 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.094926 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.094931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.095092 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.095277 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.095510 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":350,"threads_started":1}
I20260807 04:22:59.095775 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.080471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.1.244.254:39343)
0807 04:22:59.081686 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.081690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.081705 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:22:59.081751 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.081754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.081808 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.081940 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.083205 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.084020 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.085411 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.086080 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.086210 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.089374 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.089383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.089398 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.089441 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.093234 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.094072 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.094075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.094077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.094209 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.094524 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.094527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.094529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.094811 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.094922 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.095349 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.095538 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1121,"thread_start_us":1011,"threads_started":1}
I20260807 04:22:59.097443 8828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076578, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260807 04:22:59.108855 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.098033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:42132)
0807 04:22:59.098199 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.098214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.098286 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.098532 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.098535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.098825 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.098956 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.098963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.099827 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.099830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.100706 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.100717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.101346 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.101351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.101501 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.102008 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.102032 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.104227 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.106927 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.106930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.106933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.107263 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.107599 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.107603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.107605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.107722 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.108189 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.108192 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.108297 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.108413 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.108599 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 04:22:59.109027 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.098069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.1.244.253:38199)
0807 04:22:59.098225 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.098229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.098244 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:22:59.098370 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.098373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.098425 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.098499 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.099064 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.099710 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.100821 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.101221 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.101315 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.104370 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.104374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.104377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.104403 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.106807 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.107356 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.107360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.107362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.107499 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.107808 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.107811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.107813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.108094 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.108198 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.108489 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.108715 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":50}
I20260807 04:22:59.113428 8828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Bootstrap starting.
I20260807 04:22:59.116430 8828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:59.117375 8828 log.cc:824] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:59.119760 8828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: No bootstrap required, opened a new log
I20260807 04:22:59.126793 8828 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } }
I20260807 04:22:59.127079 8828 raft_consensus.cc:374] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:59.127144 8828 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47bccfe2ef84e068685777520783522, State: Initialized, Role: FOLLOWER
I20260807 04:22:59.127614 8828 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } }
I20260807 04:22:59.128424 8835 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } } }
I20260807 04:22:59.128715 8835 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:22:59.129403 8828 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:22:59.138113 8846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:22:59.138255 8846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:22:59.140017 8761 master_runner.cc:428] Detected that this master d47bccfe2ef84e068685777520783522 is joining an existing cluster
I20260807 04:22:59.140127 8761 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.244.253:38199
I20260807 04:22:59.168612 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.144059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.1.244.254:39343)
0807 04:22:59.144527 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.144537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.148576 (+ 4039us) server_negotiation.cc:407] Connection header received
0807 04:22:59.148849 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.148855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.148953 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.149030 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.150890 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.152396 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.155084 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.155996 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.156118 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.161286 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.161293 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.161300 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.161341 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.165017 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.165837 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.165842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.165846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.166020 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.166508 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.166513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.166516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.167174 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.167319 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.168012 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.168259 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260807 04:22:59.170018 8853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.144159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:39612)
0807 04:22:59.148484 (+ 4325us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.148506 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.148686 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.149294 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.149303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.150166 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.150681 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.150699 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.152617 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.152625 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.154881 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.154898 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.156483 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.156492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.156737 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.159714 (+ 2977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.159738 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.161061 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.165196 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.165201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.165206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.165687 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.166172 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.166177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.166184 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.166370 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.167426 (+ 1056us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.167431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.167643 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.168867 (+ 1224us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.169480 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4147,"thread_start_us":3953,"threads_started":1}
WARNING: no policy specified for oryx/127.1.244.252@KRBTEST.COM; defaulting to no policy
I20260807 04:22:59.176080 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.148685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.1.244.253:38199)
0807 04:22:59.148854 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.148859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.159547 (+ 10688us) server_negotiation.cc:407] Connection header received
0807 04:22:59.159809 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.159816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.159936 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.160011 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.161575 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.162788 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.164923 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.165713 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.165824 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.168467 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.168476 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.168484 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.168525 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.172760 (+ 4235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.173643 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.173650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.173656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.173820 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.174284 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.174289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.174292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.174672 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.174852 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.175392 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.175679 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:22:59.177603 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.148969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:42136)
0807 04:22:59.159416 (+ 10447us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.159436 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.159562 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.160217 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.160224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.161078 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.161425 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.161440 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.163013 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.163020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.164766 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.164787 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.166122 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.166130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.166354 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.166963 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.166993 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.168273 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.172938 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.172945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.172951 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.173493 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.173976 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.173980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.173983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.174169 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.174939 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.174944 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.175137 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.176419 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.176880 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10296,"thread_start_us":10651,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.252@KRBTEST.COM" created.
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260807 04:22:59.193614 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.177425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:42146)
0807 04:22:59.178062 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.178098 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.178314 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.178777 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.178784 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.179606 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.179829 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.179838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.180924 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.180930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.181893 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.181914 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.182634 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.182640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.182826 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.183431 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.183448 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.186067 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.190459 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.190472 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.190480 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.191050 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.191503 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.191508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.191512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.191689 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.192400 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.192410 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.192618 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.192844 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.193172 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":269,"threads_started":1}
I20260807 04:22:59.193660 8831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.177392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.1.244.253:38199)
0807 04:22:59.177565 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.177570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.178187 (+ 617us) server_negotiation.cc:407] Connection header received
0807 04:22:59.178528 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.178532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.178615 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.178727 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.180051 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.180781 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.182042 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.182472 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.182613 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.186273 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.186282 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.186288 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.186333 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.190308 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.191185 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.191193 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.191198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.191370 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.191806 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.191814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.191820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.192250 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.192409 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.192974 (+ 565us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.193223 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:22:59.195992 8290 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.244.253:38199: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d47bccfe2ef84e068685777520783522"
member_type: NON_VOTER
last_known_addr {
host: "127.1.244.253"
port: 38199
}
attrs {
promote: true
}
}
dest_uuid: "0f454652f335495597b168618f546d2a"
cas_config_opid_index: -1
I20260807 04:22:59.196913 8290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } }
I20260807 04:22:59.204677 8835 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:22:59.205049 8835 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } }
I20260807 04:22:59.206848 8835 leader_election.cc:288] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343)
I20260807 04:22:59.208031 8300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d47bccfe2ef84e068685777520783522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f454652f335495597b168618f546d2a" is_pre_election: true
I20260807 04:22:59.209471 8777 leader_election.cc:302] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d47bccfe2ef84e068685777520783522; no voters: 0f454652f335495597b168618f546d2a
I20260807 04:22:59.210317 8835 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 04:22:59.212045 8859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d47bccfe2ef84e068685777520783522 (127.1.244.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } }
I20260807 04:22:59.213064 8803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:59.216176 8859 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.244.253:38199
I20260807 04:22:59.217070 8761 master.cc:537] Master@127.1.244.253:38199 shutting down...
I20260807 04:22:59.217013 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } }
I20260807 04:22:59.217213 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:59.218717 8803 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 04:22:59.219576 8859 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.252.keytab.
Entry for principal oryx/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.252.keytab.
I20260807 04:22:59.224490 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43429
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.252.keytab
--principal=oryx/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:22:59.224691 8273 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 04:22:59.233819 8761 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:59.234336 8761 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:59.234552 8761 tablet_replica.cc:330] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: stopping tablet replica
W20260807 04:22:59.238440 8777 connection.cc:441] server connection from 127.0.0.1:42146 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42146 (request call id 3) could send its response
W20260807 04:22:59.239454 8273 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a -> Peer d47bccfe2ef84e068685777520783522 (127.1.244.253:38199): Couldn't send request to peer d47bccfe2ef84e068685777520783522. Status: Network error: recv got EOF from 127.1.244.253:38199 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 04:22:59.242983 8761 master.cc:552] Master@127.1.244.253:38199 shutdown complete.
I20260807 04:22:59.243098 8761 master_runner.cc:305] Clearing existing system tablet
I20260807 04:22:59.243561 8761 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:59.243669 8761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:59.246131 8761 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:22:59.247630 8865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:59.247987 8761 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:59.248425 8761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:22:59.248531 8761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
uuid: "d47bccfe2ef84e068685777520783522"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "6da712b1c44f6025a7519dbe5a45795e"
server_key_iv: "6958b8781d13ad50e655833d3041b8df"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.251869 8761 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:22:59.256537 8761 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 04:22:59.256721 8761 log.cc:1196] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 04:22:59.257119 8761 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Deleting consensus metadata
I20260807 04:22:59.257385 8761 master_runner.cc:315] Copying system tablet from 127.1.244.254:39343
I20260807 04:22:59.260018 8761 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: tablet copy: Beginning tablet copy session from remote peer at address 127.1.244.254:39343
I20260807 04:22:59.273005 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.260777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.1.244.254:39343)
0807 04:22:59.260978 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.260987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.261481 (+ 494us) server_negotiation.cc:407] Connection header received
0807 04:22:59.261701 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.261706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.261783 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.261925 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.262854 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.263582 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.264980 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.265409 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.265547 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.267343 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.267356 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.267360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.267390 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.270312 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.270948 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.270954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.270957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.271083 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.271496 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.271502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.271505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.271795 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.271989 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.272387 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.272587 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:22:59.273041 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.260772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:39620)
0807 04:22:59.261369 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.261388 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.261545 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.261951 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.261956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.262415 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.262694 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.262708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.263722 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.263729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.264822 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.264835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.265556 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.265561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.265748 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.266429 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.266446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.267132 (+ 686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.270453 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.270459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.270464 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.270814 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.271229 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.271234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.271238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.271384 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.271945 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.271950 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.272116 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.272295 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.272586 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":344,"threads_started":1}
I20260807 04:22:59.274170 8310 tablet_copy_service.cc:140] P 0f454652f335495597b168618f546d2a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d47bccfe2ef84e068685777520783522 ({username='oryx', principal='oryx/127.1.244.253@KRBTEST.COM'} at 127.0.0.1:39620)
I20260807 04:22:59.274399 8310 tablet_copy_service.cc:161] P 0f454652f335495597b168618f546d2a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d47bccfe2ef84e068685777520783522 at {username='oryx', principal='oryx/127.1.244.253@KRBTEST.COM'} at 127.0.0.1:39620: session id = d47bccfe2ef84e068685777520783522-00000000000000000000000000000000
I20260807 04:22:59.277735 8310 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 04:22:59.279670 8761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 04:22:59.284408 8761 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: tablet copy: Starting download of 0 data blocks...
I20260807 04:22:59.284720 8761 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: tablet copy: Starting download of 1 WAL segments...
I20260807 04:22:59.287186 8761 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 04:22:59.290540 8310 tablet_copy_service.cc:342] P 0f454652f335495597b168618f546d2a: Request end of tablet copy session d47bccfe2ef84e068685777520783522-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.244.253@KRBTEST.COM'} at 127.0.0.1:39620
I20260807 04:22:59.290721 8310 tablet_copy_service.cc:433] P 0f454652f335495597b168618f546d2a: ending tablet copy session d47bccfe2ef84e068685777520783522-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d47bccfe2ef84e068685777520783522
I20260807 04:22:59.300906 8761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:59.303499 8872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:59.309719 8873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:59.319303 8875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:59.325333 8761 server_base.cc:1034] running on GCE node
I20260807 04:22:59.325683 8761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:59.326398 8761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:59.327612 8761 hybrid_clock.cc:648] HybridClock initialized: now 1786076579327564 us; error 26 us; skew 500 ppm
I20260807 04:22:59.327848 8761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076579, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:59.329727 8761 init.cc:377] Logged in from keytab as oryx/127.1.244.253@KRBTEST.COM (short username oryx)
I20260807 04:22:59.331290 8761 webserver.cc:460] Webserver started at http://127.1.244.253:39253/ using document root <none> and password file <none>
I20260807 04:22:59.331661 8761 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:59.331758 8761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:59.334239 8761 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 04:22:59.336133 8882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:59.336565 8761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 04:22:59.336719 8761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
uuid: "d47bccfe2ef84e068685777520783522"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "6da712b1c44f6025a7519dbe5a45795e"
server_key_iv: "6958b8781d13ad50e655833d3041b8df"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.336894 8761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:59.354827 8761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:59.358305 8761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:59.391754 8761 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:38199
I20260807 04:22:59.391789 8935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:38199 every 8 connection(s)
I20260807 04:22:59.392730 8761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
I20260807 04:22:59.399689 8936 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:22:59.402379 8936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Bootstrap starting.
I20260807 04:22:59.415509 8936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:22:59.416054 8936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: Bootstrap complete.
I20260807 04:22:59.416961 8936 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } }
I20260807 04:22:59.417217 8936 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d47bccfe2ef84e068685777520783522, State: Initialized, Role: LEARNER
I20260807 04:22:59.417490 8936 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } }
I20260807 04:22:59.418283 8939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } }
I20260807 04:22:59.418596 8936 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:59.418707 8939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:22:59.426250 8950 catalog_manager.cc:1259] Loaded cluster ID: fd18d5a3c3a34e82b9af0a0b9bbc26d1
I20260807 04:22:59.426438 8950 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: loading cluster ID for follower catalog manager: success
I20260807 04:22:59.429009 8950 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: acquiring CA information for follower catalog manager: success
I20260807 04:22:59.431234 8950 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:22:59.565917 8756 heartbeater.cc:503] Master 127.1.244.254:39343 was elected leader, sending a full tablet report...
W20260807 04:22:59.646565 8863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:22:59.646826 8863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:22:59.646919 8863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:22:59.654182 8863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:22:59.654284 8863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:22:59.654330 8863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:22:59.654385 8863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:22:59.654429 8863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:22:59.663821 8863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.252.keytab
--principal=oryx/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:22:59.665041 8863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:22:59.666255 8863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:22:59.678052 8955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:59.678020 8958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:22:59.678201 8956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:22:59.679025 8863 server_base.cc:1034] running on GCE node
I20260807 04:22:59.679811 8863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:22:59.681173 8863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:22:59.682364 8863 hybrid_clock.cc:648] HybridClock initialized: now 1786076579682304 us; error 54 us; skew 500 ppm
I20260807 04:22:59.682651 8863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076579, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:22:59.685777 8863 init.cc:377] Logged in from keytab as oryx/127.1.244.252@KRBTEST.COM (short username oryx)
I20260807 04:22:59.687327 8863 webserver.cc:460] Webserver started at http://127.1.244.252:44365/ using document root <none> and password file <none>
I20260807 04:22:59.688099 8863 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:59.688232 8863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:59.688556 8863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:22:59.691577 8863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/instance:
uuid: "bd6b742437f84714b619c787d28a1df3"
format_stamp: "Formatted at 2026-08-07 04:22:59 on dist-test-slave-zkpd"
server_key: "2781c4ba97e5950a504e28339051f5f6"
server_key_iv: "3797d0a4f65ec70b33c5ebb1ce615a43"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.692381 8863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal/instance:
uuid: "bd6b742437f84714b619c787d28a1df3"
format_stamp: "Formatted at 2026-08-07 04:22:59 on dist-test-slave-zkpd"
server_key: "2781c4ba97e5950a504e28339051f5f6"
server_key_iv: "3797d0a4f65ec70b33c5ebb1ce615a43"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.697544 8863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:22:59.700922 8965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:59.702164 8863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:22:59.702338 8863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
uuid: "bd6b742437f84714b619c787d28a1df3"
format_stamp: "Formatted at 2026-08-07 04:22:59 on dist-test-slave-zkpd"
server_key: "2781c4ba97e5950a504e28339051f5f6"
server_key_iv: "3797d0a4f65ec70b33c5ebb1ce615a43"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.702520 8863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:59.724421 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.717688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:42154)
0807 04:22:59.718239 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.718263 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.718434 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.719052 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.719058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.719092 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:22:59.719316 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.719325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.720927 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.720933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.722334 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.722348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.723540 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.723547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.723721 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.723783 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.723920 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.724063 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":341,"threads_started":1}
I20260807 04:22:59.724406 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.717901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.1.244.253:38199)
0807 04:22:59.718665 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.718669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.718683 (+ 14us) server_negotiation.cc:407] Connection header received
0807 04:22:59.718799 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.718807 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.718877 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.719012 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:22:59.719508 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.720791 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.722514 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.723360 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.723571 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.723820 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.723950 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.724074 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":482,"threads_started":1}
I20260807 04:22:59.725570 8910 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 04:22:59.726104 8939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f454652f335495597b168618f546d2a. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } }
I20260807 04:22:59.726343 8939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:22:59.730319 8863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:22:59.734138 8863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:22:59.734417 8863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:22:59.752314 8863 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:43429
I20260807 04:22:59.752399 9021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:43429 every 8 connection(s)
I20260807 04:22:59.753563 8863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
I20260807 04:22:59.757148 9022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:22:59.758826 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 8863
I20260807 04:22:59.765966 9022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076579, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:22:59.782160 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.761138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.1.244.252:43429)
0807 04:22:59.761913 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.761925 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.761970 (+ 45us) server_negotiation.cc:407] Connection header received
0807 04:22:59.763600 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.763623 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.764101 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.764484 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.765995 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.767333 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.769057 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.769850 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.770063 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.774290 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.774298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.774309 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.774357 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.777886 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.778575 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.778585 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.778594 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.778808 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.779181 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.779186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.779190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.779868 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.780205 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.780527 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.780932 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":354,"threads_started":1}
I20260807 04:22:59.783789 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.768551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.1.244.254:39343)
0807 04:22:59.768779 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.768788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.769505 (+ 717us) server_negotiation.cc:407] Connection header received
0807 04:22:59.769714 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.769719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.769795 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.769901 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.771177 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.771940 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.773246 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.773769 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.773905 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.777795 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.777800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.777803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.777833 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.781119 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.781865 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.781870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.781873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.782080 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.782511 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.782515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.782517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.782784 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.782942 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.783409 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.783572 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:22:59.784091 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.768490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:39626)
0807 04:22:59.769389 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.769425 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.769586 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.769972 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.769978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.770738 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.771028 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.771037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.772069 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.772074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.773096 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.773108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.773998 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.774006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.774160 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.774822 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.774856 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.777630 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.781245 (+ 3615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.781254 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.781259 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.781694 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.782206 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.782209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.782211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.782383 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.782925 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.782932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.783094 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.783310 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.783633 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":390,"threads_started":1}
I20260807 04:22:59.785364 9022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076579, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260807 04:22:59.797529 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.785940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:42156)
0807 04:22:59.786101 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.786134 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.786228 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.786559 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.786563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.786878 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.786999 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.787007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.788048 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.788056 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.788962 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.788973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.789696 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.789701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.789885 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.790487 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.790503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.792386 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.795066 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.795072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.795076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.795437 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.795822 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.795826 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.795830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.796036 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.796621 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.796626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.796767 (+ 141us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.796901 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.797191 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 04:22:59.797545 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.786079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42156 (local address 127.1.244.253:38199)
0807 04:22:59.786263 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.786271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.786291 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:22:59.786347 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.786352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.786423 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.786541 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.787160 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.787858 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.789108 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.789550 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.789688 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.792576 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.792581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.792584 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.792619 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.794934 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.795561 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.795566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.795569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.795701 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.796145 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.796151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.796155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.796491 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.796655 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.797015 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.797230 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:22:59.798530 9022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076579, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
I20260807 04:22:59.810094 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.799306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.0.0.1:42478)
0807 04:22:59.799474 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.799494 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.799586 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.799825 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.799828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.800177 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.800310 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.800318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.801155 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.801158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.801922 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.801932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.802509 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.802513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.802671 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.803191 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.803209 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.805100 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.808145 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.808149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.808151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.808490 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.808800 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.808803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.808805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.808946 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.809383 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.809387 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.809506 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.809612 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.809796 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 04:22:59.810168 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.799251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.1.244.252:43429)
0807 04:22:59.799411 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.799415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.799563 (+ 148us) server_negotiation.cc:407] Connection header received
0807 04:22:59.799665 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.799668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.799718 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.799789 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.800427 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.801052 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.802031 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.802390 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.802483 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.805253 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.805257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.805260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.805286 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.808031 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.808583 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.808586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.808588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.808697 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.809032 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.809035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.809037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.809263 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.809385 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.809709 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.809921 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":49}
I20260807 04:22:59.814728 9022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Bootstrap starting.
I20260807 04:22:59.817778 9022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Neither blocks nor log segments found. Creating new log.
I20260807 04:22:59.818786 9022 log.cc:824] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Log is configured to *not* fsync() on all Append() calls
I20260807 04:22:59.821487 9022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: No bootstrap required, opened a new log
I20260807 04:22:59.828511 9022 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } }
I20260807 04:22:59.828787 9022 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:22:59.828881 9022 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd6b742437f84714b619c787d28a1df3, State: Initialized, Role: FOLLOWER
I20260807 04:22:59.829514 9022 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } }
I20260807 04:22:59.830287 9028 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } } }
I20260807 04:22:59.830673 9028 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:22:59.831398 9022 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:22:59.836928 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8353
W20260807 04:22:59.840130 9039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:22:59.840251 9039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:22:59.845393 8863 master_runner.cc:428] Detected that this master bd6b742437f84714b619c787d28a1df3 is joining an existing cluster
I20260807 04:22:59.845500 8863 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.244.252:43429
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.193@KRBTEST.COM
I20260807 04:22:59.865829 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.838844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.1.244.193:44963)
0807 04:22:59.841808 (+ 2964us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.841817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.841845 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:22:59.841921 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.841930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.842173 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.842394 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.844049 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.845580 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.847880 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.848747 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.848883 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.856301 (+ 7418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.856318 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.856338 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.856391 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.861229 (+ 4838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.862123 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.862131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.862135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.862278 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.862774 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.862781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.862787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.863354 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.864954 (+ 1600us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.865110 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.865363 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2799,"thread_start_us":2651,"threads_started":1}
I20260807 04:22:59.867179 8478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:22:59.875962 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.850120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.0.0.1:42484)
0807 04:22:59.859810 (+ 9690us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.859833 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.860030 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.860460 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.860466 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.861158 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.861507 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.861518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.863024 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.863029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.864611 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.864627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.865873 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.865883 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.866303 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.866949 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.866995 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.868427 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.872866 (+ 4439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.872870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.872874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.873366 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.873860 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.873864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.873868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.874042 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.874737 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.874742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.874897 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.875062 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.875439 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9560,"thread_start_us":8871,"threads_started":1}
I20260807 04:22:59.876960 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.850620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.1.244.252:43429)
0807 04:22:59.850791 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.850797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.859936 (+ 9139us) server_negotiation.cc:407] Connection header received
0807 04:22:59.860144 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.860148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.860227 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.860289 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.861665 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.862802 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.864754 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.865441 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.865551 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.868631 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.868640 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.868647 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.868686 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.872739 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.873508 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.873513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.873517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.873701 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.874155 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.874160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.874164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.874551 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.874676 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.876314 (+ 1638us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.876549 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 04:22:59.890123 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.850600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.1.244.254:39343)
0807 04:22:59.850821 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.850829 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.859486 (+ 8657us) server_negotiation.cc:407] Connection header received
0807 04:22:59.861077 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.861083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.861174 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.861248 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.864162 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.865250 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.871062 (+ 5812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.871749 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.871846 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.882301 (+ 10455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.882315 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.882321 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.882374 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.886766 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.887615 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.887621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.887625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.887797 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.888312 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.888317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.888321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.888689 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.888821 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.889498 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.889749 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:22:59.891104 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.849733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:39638)
0807 04:22:59.859366 (+ 9633us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.859392 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.860737 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.861539 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.861544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.862266 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.862647 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.862663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.865493 (+ 2830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.865499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.867154 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.867169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.872170 (+ 5001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.872178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.872362 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.874874 (+ 2512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.874898 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.877490 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.886920 (+ 9430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.886925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.886929 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.887450 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.887981 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.887986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.887990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.888174 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.888908 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.888913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.889141 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.890308 (+ 1167us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.890642 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9454,"thread_start_us":9759,"threads_started":1}
I20260807 04:22:59.900043 8971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.860118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42160 (local address 127.1.244.253:38199)
0807 04:22:59.860338 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.860345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.870840 (+ 10495us) server_negotiation.cc:407] Connection header received
0807 04:22:59.872961 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.872967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.873068 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.873139 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.875362 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.876504 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.883143 (+ 6639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.883806 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.883932 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.893391 (+ 9459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.893398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.893403 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.893454 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.897109 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.897852 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.897858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.897862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.898009 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.898449 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.898454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.898458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.898788 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.898899 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.899474 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.899700 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 04:22:59.901039 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.860486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:42160)
0807 04:22:59.870735 (+ 10249us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.870762 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.872800 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.873357 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.873362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.874062 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.874463 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.874476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.880027 (+ 5551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.880040 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.881984 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.882004 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.885188 (+ 3184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.885195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.891474 (+ 6279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.892115 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.892138 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.893183 (+ 1045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.897265 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.897271 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.897275 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.897722 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.898144 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.898149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.898153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.898321 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.898993 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.898998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.899188 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.900299 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.900607 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10120,"thread_start_us":10373,"threads_started":1}
Aug 07 04:22:59 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
I20260807 04:22:59.914153 8970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.897162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.0.0.1:42498)
0807 04:22:59.897560 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 04:22:59.897579 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:22:59.897677 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:22:59.898567 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:22:59.898572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:22:59.899007 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:22:59.899214 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.899226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.900841 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.900848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.902224 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:22:59.902239 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.903368 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.903374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:22:59.903553 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.904180 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:22:59.904200 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:22:59.906865 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:22:59.911033 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.911039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.911043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.911568 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.911979 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:22:59.911984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:22:59.911988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:22:59.912166 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:22:59.913223 (+ 1057us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:22:59.913228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:22:59.913381 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 04:22:59.913532 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 04:22:59.913814 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283}
I20260807 04:22:59.914960 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:22:59.897889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42498 (local address 127.1.244.252:43429)
0807 04:22:59.898059 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:22:59.898065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:22:59.898082 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:22:59.898168 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:22:59.898173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:22:59.898259 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:22:59.898350 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:22:59.899383 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.900661 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.902383 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:22:59.903091 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:22:59.903200 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:22:59.907062 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:22:59.907068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:22:59.907073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:22:59.907106 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:22:59.910883 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.911703 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.911709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.911713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.911835 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:22:59.912288 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:22:59.912293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:22:59.912297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:22:59.913023 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:22:59.913148 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 04:22:59.914433 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0807 04:22:59.914635 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:22:59.916401 8290 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.244.252:43429: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bd6b742437f84714b619c787d28a1df3"
member_type: NON_VOTER
last_known_addr {
host: "127.1.244.252"
port: 43429
}
attrs {
promote: true
}
}
dest_uuid: "0f454652f335495597b168618f546d2a"
cas_config_opid_index: 5
I20260807 04:22:59.917450 8290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } }
I20260807 04:22:59.919353 8859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bd6b742437f84714b619c787d28a1df3 (127.1.244.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.920651 8910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 LEARNER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 04:22:59.922178 8996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:22:59.927206 8859 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.244.252:43429
I20260807 04:22:59.928314 9051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 04:22:59.931084 8910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bd6b742437f84714b619c787d28a1df3 (127.1.244.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.935587 8939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.935906 8939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:22:59.940712 8859 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: attempting to promote NON_VOTER d47bccfe2ef84e068685777520783522 to VOTER
I20260807 04:22:59.940817 9051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: NON_VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: true } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.941171 9051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:59.941761 8859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } }
I20260807 04:22:59.949019 8910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 LEARNER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 04:22:59.950323 9050 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 04:22:59.952508 8996 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 04:22:59.953120 8863 master.cc:537] Master@127.1.244.252:43429 shutting down...
I20260807 04:22:59.955839 8995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 04:22:59.955789 9050 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d47bccfe2ef84e068685777520783522 (127.1.244.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.959415 8910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d47bccfe2ef84e068685777520783522 (127.1.244.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.963685 8859 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:22:59.964380 8939 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.964643 8939 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:22:59.976563 8863 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:22:59.977020 8863 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:22:59.977289 8863 tablet_replica.cc:330] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: stopping tablet replica
I20260807 04:22:59.980926 9051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.981211 9051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:22:59.982197 9050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:22:59.982475 9050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
W20260807 04:22:59.982932 8274 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a -> Peer bd6b742437f84714b619c787d28a1df3 (127.1.244.252:43429): Couldn't send request to peer bd6b742437f84714b619c787d28a1df3. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 04:22:59.984661 8863 master.cc:552] Master@127.1.244.252:43429 shutdown complete.
I20260807 04:22:59.984747 8863 master_runner.cc:305] Clearing existing system tablet
I20260807 04:22:59.985131 8863 fs_manager.cc:359] Metadata directory not provided
I20260807 04:22:59.985239 8863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:22:59.988160 8863 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 04:22:59.989823 9058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:22:59.990199 8863 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:22:59.990461 8863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 04:22:59.990558 8863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
uuid: "bd6b742437f84714b619c787d28a1df3"
format_stamp: "Formatted at 2026-08-07 04:22:59 on dist-test-slave-zkpd"
server_key: "2781c4ba97e5950a504e28339051f5f6"
server_key_iv: "3797d0a4f65ec70b33c5ebb1ce615a43"
server_key_version: "encryptionkey@0"
I20260807 04:22:59.995594 8863 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:00.002511 8863 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 04:23:00.002671 8863 log.cc:1196] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 04:23:00.003175 8863 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Deleting consensus metadata
I20260807 04:23:00.003444 8863 master_runner.cc:315] Copying system tablet from 127.1.244.254:39343
I20260807 04:23:00.005992 8863 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: tablet copy: Beginning tablet copy session from remote peer at address 127.1.244.254:39343
I20260807 04:23:00.021613 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8353
I20260807 04:23:00.033258 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.006818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.1.244.254:39343)
0807 04:23:00.007045 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:00.007051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:00.012431 (+ 5380us) server_negotiation.cc:407] Connection header received
0807 04:23:00.012632 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:00.012638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:00.012725 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:00.012837 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:00.013846 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.014938 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.016807 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.017457 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.017590 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.020289 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:00.020296 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:00.020301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:00.020340 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:00.030073 (+ 9733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.030864 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.030884 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.030888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.031033 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.031509 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.031514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.031518 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.031801 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:00.032081 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:00.032524 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 04:23:00.032770 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 04:23:00.033296 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.006738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:39650)
0807 04:23:00.012289 (+ 5551us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:00.012315 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:00.012481 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:00.012882 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:00.012887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:00.013410 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:00.013661 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.013675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.015104 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.015109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.016647 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.016662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.017622 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.017629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.017824 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.018566 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:00.018590 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:00.020078 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:00.030217 (+ 10139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.030222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.030227 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.030687 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.031175 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.031180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.031184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.031387 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.032020 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:00.032037 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:00.032241 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 04:23:00.032413 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 04:23:00.032790 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5403,"thread_start_us":5304,"threads_started":1}
I20260807 04:23:00.034173 8310 tablet_copy_service.cc:140] P 0f454652f335495597b168618f546d2a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bd6b742437f84714b619c787d28a1df3 ({username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:39650)
I20260807 04:23:00.034379 8310 tablet_copy_service.cc:161] P 0f454652f335495597b168618f546d2a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bd6b742437f84714b619c787d28a1df3 at {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:39650: session id = bd6b742437f84714b619c787d28a1df3-00000000000000000000000000000000
I20260807 04:23:00.037747 8310 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 04:23:00.039786 8863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 04:23:00.046794 8863 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: tablet copy: Starting download of 0 data blocks...
I20260807 04:23:00.047091 8863 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: tablet copy: Starting download of 1 WAL segments...
I20260807 04:23:00.048184 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:44963
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:39343
--webserver_port=34073
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:23:00.049579 8863 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 04:23:00.053881 8310 tablet_copy_service.cc:342] P 0f454652f335495597b168618f546d2a: Request end of tablet copy session bd6b742437f84714b619c787d28a1df3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:39650
I20260807 04:23:00.054019 8310 tablet_copy_service.cc:433] P 0f454652f335495597b168618f546d2a: ending tablet copy session bd6b742437f84714b619c787d28a1df3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bd6b742437f84714b619c787d28a1df3
I20260807 04:23:00.066354 8863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:00.070225 9067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:00.073529 9068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:00.083592 9070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:00.089373 8863 server_base.cc:1034] running on GCE node
I20260807 04:23:00.089763 8863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:00.090430 8863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:00.091639 8863 hybrid_clock.cc:648] HybridClock initialized: now 1786076580091590 us; error 34 us; skew 500 ppm
I20260807 04:23:00.092123 8863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:00 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076580, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:00.094391 8863 init.cc:377] Logged in from keytab as oryx/127.1.244.252@KRBTEST.COM (short username oryx)
I20260807 04:23:00.095667 8863 webserver.cc:460] Webserver started at http://127.1.244.252:42611/ using document root <none> and password file <none>
I20260807 04:23:00.096017 8863 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:00.096122 8863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:00.099489 8863 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 04:23:00.101500 9077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:00.101895 8863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 04:23:00.102021 8863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
uuid: "bd6b742437f84714b619c787d28a1df3"
format_stamp: "Formatted at 2026-08-07 04:22:59 on dist-test-slave-zkpd"
server_key: "2781c4ba97e5950a504e28339051f5f6"
server_key_iv: "3797d0a4f65ec70b33c5ebb1ce615a43"
server_key_version: "encryptionkey@0"
I20260807 04:23:00.102222 8863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:00.129222 8863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:00.133590 8863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:00.163128 8863 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:43429
I20260807 04:23:00.163337 9130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:43429 every 8 connection(s)
I20260807 04:23:00.164222 8863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
I20260807 04:23:00.169657 9131 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:23:00.173966 9131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Bootstrap starting.
I20260807 04:23:00.193312 9131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:23:00.193888 9131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Bootstrap complete.
I20260807 04:23:00.195042 9131 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } }
I20260807 04:23:00.195390 9131 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bd6b742437f84714b619c787d28a1df3, State: Initialized, Role: LEARNER
I20260807 04:23:00.195679 9131 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } }
I20260807 04:23:00.196918 9131 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:00.196548 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:23:00.197217 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:23:00.204546 9145 catalog_manager.cc:1259] Loaded cluster ID: fd18d5a3c3a34e82b9af0a0b9bbc26d1
I20260807 04:23:00.204692 9145 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: loading cluster ID for follower catalog manager: success
I20260807 04:23:00.209206 9145 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: acquiring CA information for follower catalog manager: success
I20260807 04:23:00.211607 9145 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 04:23:00.430406 9066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:00.430670 9066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:00.430876 9066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:00.439632 9066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:00.439775 9066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:00.439996 9066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:00.449834 9066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:44963
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=34073
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:00.451159 9066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:00.452585 9066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:00.465492 9150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:00.465442 9151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:00.465636 9153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:00.466490 9066 server_base.cc:1034] running on GCE node
I20260807 04:23:00.467165 9066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:00.468156 9066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:00.469394 9066 hybrid_clock.cc:648] HybridClock initialized: now 1786076580469334 us; error 47 us; skew 500 ppm
I20260807 04:23:00.469712 9066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:00 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076580, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:00.473215 9066 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260807 04:23:00.474678 9066 webserver.cc:460] Webserver started at http://127.1.244.193:34073/ using document root <none> and password file <none>
I20260807 04:23:00.475389 9066 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:00.475553 9066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:00.481208 9066 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 04:23:00.483883 9160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:00.485245 9066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:00.485427 9066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "c676684f55c341c683428b058cb45faf"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "d0e20fd9042c1e9526dac6feff3babb6"
server_key_iv: "ce823327449b61a6762703561901e879"
server_key_version: "encryptionkey@0"
I20260807 04:23:00.486390 9066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:00.504070 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.497601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.0.0.1:36582)
0807 04:23:00.498077 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:00.498101 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:00.498265 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:00.498708 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:00.498719 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:00.498739 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:00.498946 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.498955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.500670 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.500673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.501846 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.501857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.503377 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.503382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.503572 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.503598 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 04:23:00.503672 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 04:23:00.503767 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":227,"threads_started":1}
I20260807 04:23:00.504366 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.497800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36582 (local address 127.1.244.252:43429)
0807 04:23:00.498351 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:00.498357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:00.498377 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:23:00.498456 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:00.498461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:00.498544 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:00.498657 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:00.499075 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.500549 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.501990 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.503173 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.503293 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.503502 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:00.503721 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 04:23:00.503851 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
I20260807 04:23:00.505466 9105 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260807 04:23:00.505999 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f454652f335495597b168618f546d2a. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:23:00.506316 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: This master's current role is: LEARNER
I20260807 04:23:00.514866 9066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:00.518940 9066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:00.519215 9066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:00.520422 9066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:00.521867 9066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:00.521991 9066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:00.522104 9066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:00.522182 9066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:00.558107 9066 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44963
I20260807 04:23:00.558202 9276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44963 every 8 connection(s)
I20260807 04:23:00.559563 9066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:00.565011 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 9066
I20260807 04:23:00.565462 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8489
Aug 07 04:23:00 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076580, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Aug 07 04:23:00 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076580, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
Aug 07 04:23:00 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.194@KRBTEST.COM
I20260807 04:23:00.600494 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.562461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:42659 (local address 127.1.244.253:38199)
0807 04:23:00.582006 (+ 19545us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:00.582014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:00.582879 (+ 865us) server_negotiation.cc:407] Connection header received
0807 04:23:00.582946 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:00.582951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:00.583039 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:00.583134 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:00.584492 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.585565 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.587534 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.588257 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.588373 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.592877 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:00.592884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:00.592889 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:00.592931 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:00.596740 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.597827 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.597835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.597842 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.598056 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.598587 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.598593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.598596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.599026 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:00.599175 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:00.599825 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 04:23:00.600110 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19413,"thread_start_us":20278,"threads_started":1}
I20260807 04:23:00.601805 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.562640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51981 (local address 127.1.244.252:43429)
0807 04:23:00.562871 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:00.562880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:00.575481 (+ 12601us) server_negotiation.cc:407] Connection header received
0807 04:23:00.579565 (+ 4084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:00.579570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:00.579675 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:00.579797 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:00.582560 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.583671 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.586313 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.586958 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.587051 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.592375 (+ 5324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:00.592382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:00.592387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:00.592435 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:00.596332 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.597430 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.597438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.597442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.597590 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.598192 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.598196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.598199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.598579 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:00.598718 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:00.600165 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0807 04:23:00.600402 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 04:23:00.603659 9283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.563040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.1.244.193:42659)
0807 04:23:00.582559 (+ 19519us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:00.582576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:00.582688 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:00.583311 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:00.583315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:00.583939 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:00.584284 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.584300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.585758 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.585763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.587348 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.587362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.588629 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.588639 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.588840 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.589483 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:00.589504 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:00.592660 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:00.596992 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.596999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.597003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.597672 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.598252 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.598259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.598264 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.598441 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.599248 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:00.599255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:00.599432 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 04:23:00.599688 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 04:23:00.602128 (+ 2440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19353,"mutex_wait_us":232,"spinlock_wait_cycles":58752,"thread_start_us":19038,"threads_started":1}
I20260807 04:23:00.603839 9282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.563040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.1.244.193:51981)
0807 04:23:00.575267 (+ 12227us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:00.575332 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:00.579269 (+ 3937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:00.580333 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:00.580349 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:00.581338 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:00.582321 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.582359 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.583921 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.583927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.586148 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.586168 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.587386 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.587395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.587687 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.588557 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:00.588604 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:00.592091 (+ 3487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:00.596498 (+ 4407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.596507 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.596524 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.597279 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.597771 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.597777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.597782 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.598070 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.598790 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:00.598799 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:00.599248 (+ 449us) client_negotiation.cc:769] Sending connection context
0807 04:23:00.600036 (+ 788us) client_negotiation.cc:240] Negotiation successful
0807 04:23:00.601930 (+ 1894us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11862,"mutex_wait_us":1562,"thread_start_us":11588,"threads_started":1}
I20260807 04:23:00.605580 9277 heartbeater.cc:347] Connected to a master server at 127.1.244.252:43429
I20260807 04:23:00.605684 9279 heartbeater.cc:347] Connected to a master server at 127.1.244.253:38199
Aug 07 04:23:00 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076580, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:00.605933 9277 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:00.606848 9277 heartbeater.cc:511] Master 127.1.244.252:43429 requested a full tablet report, sending...
I20260807 04:23:00.608223 9285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.567116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53048 (local address 127.1.244.194:42793)
0807 04:23:00.576375 (+ 9259us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:00.576398 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:00.578961 (+ 2563us) server_negotiation.cc:407] Connection header received
0807 04:23:00.579069 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:00.579081 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:00.579384 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:00.579591 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:00.586743 (+ 7152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.589721 (+ 2978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.592458 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.593677 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.593852 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.599940 (+ 6088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:00.599954 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:00.599971 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:00.600011 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:00.604225 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.605146 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.605154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.605158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.605322 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.605777 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.605786 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.605789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.606255 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:00.606579 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:00.607475 (+ 896us) server_negotiation.cc:299] Negotiation successful
0807 04:23:00.607747 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9047,"thread_start_us":8847,"threads_started":1}
I20260807 04:23:00.609133 9095 ts_manager.cc:194] Registered new tserver with Master: c676684f55c341c683428b058cb45faf (127.1.244.193:44963)
I20260807 04:23:00.609452 8614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:00.605930 9279 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:00.610162 9279 heartbeater.cc:511] Master 127.1.244.253:38199 requested a full tablet report, sending...
I20260807 04:23:00.611631 8900 ts_manager.cc:194] Registered new tserver with Master: c676684f55c341c683428b058cb45faf (127.1.244.193:44963)
I20260807 04:23:00.614689 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.582726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:32861)
0807 04:23:00.594796 (+ 12070us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:00.594815 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:00.594930 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:00.596541 (+ 1611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:00.596545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:00.597314 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:00.597944 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.597957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.599483 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.599488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.601459 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:00.601472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.602501 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.602508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:00.602730 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.603310 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:00.603331 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:00.606578 (+ 3247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:00.611675 (+ 5097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.611680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.611683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.612268 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.612778 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:00.612782 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:00.612785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:00.613008 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:00.613714 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:00.613717 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:00.613841 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 04:23:00.614000 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 04:23:00.614306 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11939,"thread_start_us":11201,"threads_started":1}
I20260807 04:23:00.615144 9287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:00.582842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:32861 (local address 127.1.244.254:39343)
0807 04:23:00.588372 (+ 5530us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:00.588378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:00.595967 (+ 7589us) server_negotiation.cc:407] Connection header received
0807 04:23:00.596043 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:00.596049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:00.596134 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:00.596242 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:00.598238 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.599354 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.601622 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:00.602244 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:00.602442 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:00.606761 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:00.606767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:00.606771 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:00.606816 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:00.611507 (+ 4691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.612410 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.612417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.612440 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.612650 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:00.613121 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:00.613129 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:00.613135 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:00.613580 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:00.613784 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:00.614154 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 04:23:00.614728 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5388,"thread_start_us":5560,"threads_started":1}
I20260807 04:23:00.616000 9278 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39343
I20260807 04:23:00.616154 9278 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:00.616597 9278 heartbeater.cc:511] Master 127.1.244.254:39343 requested a full tablet report, sending...
I20260807 04:23:00.617738 8290 ts_manager.cc:194] Re-registered known tserver with Master: c676684f55c341c683428b058cb45faf (127.1.244.193:44963)
I20260807 04:23:00.618547 8290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:32861
I20260807 04:23:00.699092 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8489
I20260807 04:23:00.718708 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:42793
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:39343
--webserver_port=42445
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:23:00.907042 9050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: NON_VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: true } } }
I20260807 04:23:00.907500 9050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:00.909350 9050 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: attempting to promote NON_VOTER bd6b742437f84714b619c787d28a1df3 to VOTER
I20260807 04:23:00.909937 9050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:00.911772 8910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 04:23:00.911962 9105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 LEARNER]: Refusing update from remote peer 0f454652f335495597b168618f546d2a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 04:23:00.912492 9291 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 04:23:00.912938 9290 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 04:23:00.916178 9050 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bd6b742437f84714b619c787d28a1df3 (127.1.244.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.920670 8910 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bd6b742437f84714b619c787d28a1df3 (127.1.244.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.921190 9050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.921927 9050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:00.922171 9050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.922485 9050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:00.926422 9295 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.926899 9295 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:23:00.930876 9290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.931228 9290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:00.931895 9105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bd6b742437f84714b619c787d28a1df3 (127.1.244.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.936309 9134 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f454652f335495597b168618f546d2a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:00.936630 9134 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 04:23:01.056241 9289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:01.056499 9289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:01.056656 9289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:01.063870 9289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:01.063980 9289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:01.064177 9289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:01.073451 9289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:42793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=42445
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:01.074674 9289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:01.076136 9289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:01.088881 9302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:01.088999 9304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:01.089367 9301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:01.089658 9289 server_base.cc:1034] running on GCE node
I20260807 04:23:01.090312 9289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:01.091127 9289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:01.092316 9289 hybrid_clock.cc:648] HybridClock initialized: now 1786076581092271 us; error 36 us; skew 500 ppm
I20260807 04:23:01.092625 9289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:01.095742 9289 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260807 04:23:01.097298 9289 webserver.cc:460] Webserver started at http://127.1.244.194:42445/ using document root <none> and password file <none>
I20260807 04:23:01.097975 9289 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:01.098132 9289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:01.103860 9289 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:23:01.106468 9311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:01.107750 9289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:23:01.108006 9289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "7b8f0386037d4c9aa7e6d041a2de9c57"
format_stamp: "Formatted at 2026-08-07 04:22:57 on dist-test-slave-zkpd"
server_key: "86a7d51476bea0d0346ee05ff0dd3754"
server_key_iv: "9045b94cb02902ba088fadabe63be716"
server_key_version: "encryptionkey@0"
I20260807 04:23:01.108949 9289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:01.126384 9289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:01.130416 9289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:01.130736 9289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:01.131630 9289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:01.132954 9289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:01.133039 9289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:01.133118 9289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:01.133172 9289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:01.165680 9289 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:42793
I20260807 04:23:01.165864 9427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:42793 every 8 connection(s)
I20260807 04:23:01.167053 9289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:01.173309 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 9289
I20260807 04:23:01.173619 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8624
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.195@KRBTEST.COM
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
I20260807 04:23:01.222810 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.169978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:37853 (local address 127.1.244.252:43429)
0807 04:23:01.200887 (+ 30909us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.200893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.200922 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:23:01.200983 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.200988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.201060 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.201167 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.202969 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.204556 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.207768 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.208456 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.208568 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.214829 (+ 6261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.214836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.214841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.214881 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.218487 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.219416 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.219422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.219425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.219602 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.220219 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.220225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.220230 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.220565 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.220703 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.222149 (+ 1446us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.222384 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30770,"thread_start_us":30594,"threads_started":1}
I20260807 04:23:01.224062 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.169736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:50327 (local address 127.1.244.254:39343)
0807 04:23:01.191025 (+ 21289us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.191034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.194986 (+ 3952us) server_negotiation.cc:407] Connection header received
0807 04:23:01.199466 (+ 4480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.199472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.199553 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.199682 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.206282 (+ 6600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.207375 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.209485 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.210115 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.210226 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.214787 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.214794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.214801 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.214851 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.219620 (+ 4769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.220487 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.220495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.220499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.220703 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.221250 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.221255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.221257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.221605 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.223267 (+ 1662us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.223452 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.223669 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21126,"thread_start_us":24534,"threads_started":1}
I20260807 04:23:01.224205 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.170461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.1.244.194:37853)
0807 04:23:01.191639 (+ 21178us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.191686 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.199025 (+ 7339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.201402 (+ 2377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.201406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.201837 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:01.202660 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.202708 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.204794 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.204803 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.207563 (+ 2760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.207578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.208894 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.208914 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.209317 (+ 403us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.210051 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:01.210082 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:01.214539 (+ 4457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:01.218639 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.218647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.218661 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.219262 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.219742 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.219748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.219753 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.220071 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.220795 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:01.220804 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:01.221337 (+ 533us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.221652 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.223265 (+ 1613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20863,"mutex_wait_us":1,"thread_start_us":20622,"threads_started":1}
I20260807 04:23:01.225243 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.174650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.1.244.195:35953)
0807 04:23:01.182487 (+ 7837us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.182499 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.187760 (+ 5261us) server_negotiation.cc:407] Connection header received
0807 04:23:01.188841 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.188854 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.189145 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.189396 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.196402 (+ 7006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.198231 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.207442 (+ 9211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.208323 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.214454 (+ 6131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.216335 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.216353 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.216375 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.216422 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.221110 (+ 4688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.221948 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.221958 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.221961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.222130 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.222605 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.222609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.222612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.223335 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.224366 (+ 1031us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.224534 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.224795 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7638,"thread_start_us":7466,"threads_started":1}
I20260807 04:23:01.225495 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.171069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:50327)
0807 04:23:01.194869 (+ 23800us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.194883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.199298 (+ 4415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.200113 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.200124 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.201209 (+ 1085us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:01.206112 (+ 4903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.206124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.207566 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.207572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.209322 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.209335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.210458 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.210466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.210644 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.211275 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:01.211297 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:01.214541 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:01.219776 (+ 5235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.219782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.219787 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.220343 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.220872 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.220879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.220883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.221115 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.221776 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:01.221780 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:01.222007 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.222197 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.222554 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23678,"mutex_wait_us":67,"thread_start_us":23485,"threads_started":1}
I20260807 04:23:01.225972 9428 heartbeater.cc:347] Connected to a master server at 127.1.244.252:43429
I20260807 04:23:01.226289 9428 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:01.227032 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.201752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.1.244.194:51435)
0807 04:23:01.205412 (+ 3660us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.205437 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.205567 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.208269 (+ 2702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.208274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.208862 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:01.209249 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.209260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.210688 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.210692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.212518 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.212535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.214739 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.214745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.214926 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.215521 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:01.215546 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:01.218472 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:01.222728 (+ 4256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.222733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.222736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.224292 (+ 1556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.224722 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.224728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.224731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.224922 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.226049 (+ 1127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:01.226053 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:01.226224 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.226354 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.226628 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3498,"thread_start_us":3321,"threads_started":1}
I20260807 04:23:01.227157 8750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:01.227218 9428 heartbeater.cc:511] Master 127.1.244.252:43429 requested a full tablet report, sending...
I20260807 04:23:01.227263 9429 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39343
I20260807 04:23:01.227389 9429 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:01.227849 9429 heartbeater.cc:511] Master 127.1.244.254:39343 requested a full tablet report, sending...
I20260807 04:23:01.227963 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.201843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:51435 (local address 127.1.244.253:38199)
0807 04:23:01.207782 (+ 5939us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.207789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.207807 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:23:01.207868 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.207872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.208002 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.208104 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.209437 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.210509 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.212679 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.213503 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.213640 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.218730 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.218740 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.218747 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.218809 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.222555 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.224445 (+ 1890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.224450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.224454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.224608 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.225066 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.225071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.225076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.225824 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.225979 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.227302 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.227546 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5793,"thread_start_us":12423,"threads_started":1}
I20260807 04:23:01.228962 8290 ts_manager.cc:194] Re-registered known tserver with Master: 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793)
I20260807 04:23:01.229205 9095 ts_manager.cc:194] Registered new tserver with Master: 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793)
I20260807 04:23:01.230012 8290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:50327
I20260807 04:23:01.230521 9430 heartbeater.cc:347] Connected to a master server at 127.1.244.253:38199
I20260807 04:23:01.230708 9430 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:01.231307 9430 heartbeater.cc:511] Master 127.1.244.253:38199 requested a full tablet report, sending...
I20260807 04:23:01.232754 8900 ts_manager.cc:194] Registered new tserver with Master: 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793)
I20260807 04:23:01.317530 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8624
I20260807 04:23:01.336925 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:35953
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:39343
--webserver_port=32895
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:23:01.621721 9278 heartbeater.cc:503] Master 127.1.244.254:39343 was elected leader, sending a full tablet report...
W20260807 04:23:01.677776 9441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:01.678081 9441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:01.678273 9441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:01.685490 9441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:01.685640 9441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:01.685855 9441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:01.695544 9441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:35953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=32895
--tserver_master_addrs=127.1.244.254:39343,127.1.244.253:38199,127.1.244.252:43429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:01.696993 9441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:01.698424 9441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:01.713083 9447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:01.713166 9450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:01.713127 9448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:01.713811 9441 server_base.cc:1034] running on GCE node
I20260807 04:23:01.714435 9441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:01.715332 9441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:01.716522 9441 hybrid_clock.cc:648] HybridClock initialized: now 1786076581716456 us; error 56 us; skew 500 ppm
I20260807 04:23:01.716782 9441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:01.720295 9441 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260807 04:23:01.721872 9441 webserver.cc:460] Webserver started at http://127.1.244.195:32895/ using document root <none> and password file <none>
I20260807 04:23:01.722643 9441 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:01.722819 9441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:01.728720 9441 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 04:23:01.731609 9457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:01.733006 9441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:23:01.733194 9441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7"
format_stamp: "Formatted at 2026-08-07 04:22:58 on dist-test-slave-zkpd"
server_key: "708723687b6f27f05ae12cd5ada3ef66"
server_key_iv: "ea21568ef04d88b3822d2bc80eca185e"
server_key_version: "encryptionkey@0"
I20260807 04:23:01.734251 9441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:01.762523 9441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:01.767227 9441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:01.767547 9441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:01.768589 9441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:01.770074 9441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:01.770206 9441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:01.770352 9441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:01.770452 9441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:01.806701 9441 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:35953
I20260807 04:23:01.806794 9571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:35953 every 8 connection(s)
I20260807 04:23:01.808323 9441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:01.810873 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 9441
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.252@KRBTEST.COM
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.253@KRBTEST.COM
Aug 07 04:23:01 dist-test-slave-zkpd krb5kdc[8243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076581, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:01.872140 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.811925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42105 (local address 127.1.244.252:43429)
0807 04:23:01.836411 (+ 24486us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.836421 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.836451 (+ 30us) server_negotiation.cc:407] Connection header received
0807 04:23:01.838879 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.838887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.839128 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.839238 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.841736 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.843489 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.846475 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.847282 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.848250 (+ 968us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.862047 (+ 13797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.862055 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.862062 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.862106 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.867445 (+ 5339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.868647 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.868656 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.868662 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.868847 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.869485 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.869494 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.869501 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.870263 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.870442 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.871424 (+ 982us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.871672 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24302,"thread_start_us":24157,"threads_started":1}
I20260807 04:23:01.873849 9581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.812157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.1.244.195:42105)
0807 04:23:01.820051 (+ 7894us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.820118 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.835586 (+ 15468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.839497 (+ 3911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.839503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.840568 (+ 1065us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:01.841483 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.841504 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.843784 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.843793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.846220 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.846241 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.847544 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.847552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.847826 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.852476 (+ 4650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:01.852518 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:01.859285 (+ 6767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:01.867654 (+ 8369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.867667 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.867690 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.868457 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.868998 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.869003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.869007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.869320 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.870436 (+ 1116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:01.870450 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:01.870946 (+ 496us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.871286 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.871949 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7425,"mutex_wait_us":38,"spinlock_wait_cycles":22272,"thread_start_us":25890,"threads_started":1}
I20260807 04:23:01.876010 9580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.812396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.1.244.195:33411)
0807 04:23:01.837051 (+ 24655us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.837078 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.837242 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.838964 (+ 1722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.838979 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.842739 (+ 3760us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:01.843114 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.843126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.845427 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.845432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.846684 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.846696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.847733 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.847740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.847980 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.855049 (+ 7069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:01.855074 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:01.859285 (+ 4211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:01.870181 (+ 10896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.870188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.870194 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.870718 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.872167 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.872171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.872174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.872725 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.874168 (+ 1443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:01.874172 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:01.874330 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.874473 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.874750 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24467,"mutex_wait_us":4,"thread_start_us":7034,"threads_started":1}
I20260807 04:23:01.876029 9584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.821004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47877 (local address 127.1.244.254:39343)
0807 04:23:01.832286 (+ 11282us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.832296 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.838165 (+ 5869us) server_negotiation.cc:407] Connection header received
0807 04:23:01.838449 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.838454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.838536 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.838640 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.841646 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.842549 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.845377 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.846041 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.846164 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.862809 (+ 16645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.862816 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.862820 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.862865 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.866944 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.870530 (+ 3586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.870537 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.870540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.870710 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.872970 (+ 2260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.872979 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.872984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.873394 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.873560 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.875355 (+ 1795us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.875599 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11078,"thread_start_us":11283,"threads_started":1}
I20260807 04:23:01.876055 9572 heartbeater.cc:347] Connected to a master server at 127.1.244.252:43429
I20260807 04:23:01.876621 9572 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:01.877331 9577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.811369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:33411 (local address 127.1.244.253:38199)
0807 04:23:01.838290 (+ 26921us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.838295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.838314 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:23:01.838381 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.838387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.838469 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.838599 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.843261 (+ 4662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.844243 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.846823 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.847457 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.847616 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.859522 (+ 11906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.859533 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.859540 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.859607 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.864085 (+ 4478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.870890 (+ 6805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.870896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.870902 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.871081 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.873636 (+ 2555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.873642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.873646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.874041 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.874208 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.876438 (+ 2230us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.876703 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26748,"thread_start_us":26622,"threads_started":1}
I20260807 04:23:01.877506 9572 heartbeater.cc:511] Master 127.1.244.252:43429 requested a full tablet report, sending...
I20260807 04:23:01.879585 9095 ts_manager.cc:194] Registered new tserver with Master: de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953)
I20260807 04:23:01.879660 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.820804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:47877)
0807 04:23:01.838022 (+ 17218us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.838044 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.838191 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.839230 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.839234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.840533 (+ 1299us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:01.841466 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.841485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.842697 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.842702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.845222 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.845237 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.846950 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.846956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.847251 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.856057 (+ 8806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:01.856079 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:01.862572 (+ 6493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:01.869488 (+ 6916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.869495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.869501 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.870136 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.872349 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:01.872353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:01.872356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:01.872814 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:01.874284 (+ 1470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:01.874292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:01.874492 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.874643 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.874944 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17053,"mutex_wait_us":140,"thread_start_us":16842,"threads_started":1}
I20260807 04:23:01.880069 9575 heartbeater.cc:347] Connected to a master server at 127.1.244.253:38199
I20260807 04:23:01.880209 9575 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:01.880693 9575 heartbeater.cc:511] Master 127.1.244.253:38199 requested a full tablet report, sending...
I20260807 04:23:01.881868 8900 ts_manager.cc:194] Registered new tserver with Master: de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953)
I20260807 04:23:01.882838 9574 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39343
I20260807 04:23:01.882969 9574 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:01.883432 9574 heartbeater.cc:511] Master 127.1.244.254:39343 requested a full tablet report, sending...
I20260807 04:23:01.884443 8290 ts_manager.cc:194] Re-registered known tserver with Master: de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953)
I20260807 04:23:01.885679 8290 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:47877
I20260807 04:23:01.893432 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.849741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.1.244.252:43429)
0807 04:23:01.861556 (+ 11815us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.861560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.867834 (+ 6274us) server_negotiation.cc:407] Connection header received
0807 04:23:01.868136 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.868142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.868227 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.868308 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.872839 (+ 4531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.874206 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.876770 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.877528 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.877688 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.882018 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.882024 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.882029 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.882064 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.889766 (+ 7702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.890948 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.890952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.890955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.891141 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.891632 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.891636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.891639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.892043 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.892283 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.892812 (+ 529us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.893036 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11706,"thread_start_us":11603,"threads_started":1}
I20260807 04:23:01.895376 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.849447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.1.244.254:39343)
0807 04:23:01.849987 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.849995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.863715 (+ 13720us) server_negotiation.cc:407] Connection header received
0807 04:23:01.863784 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.863790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.863884 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.864026 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.867570 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.869036 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.877703 (+ 8667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.883491 (+ 5788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.883709 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.887637 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.887643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.887647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.887680 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.892266 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.893209 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.893217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.893222 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.893366 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.893797 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.893802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.893805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.894140 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.894393 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.894763 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.895016 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":477,"threads_started":1}
I20260807 04:23:01.903824 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.862337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.1.244.253:38199)
0807 04:23:01.876653 (+ 14316us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.876659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.878570 (+ 1911us) server_negotiation.cc:407] Connection header received
0807 04:23:01.878629 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.878635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.878705 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.878801 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:01.884358 (+ 5557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.887393 (+ 3035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.889439 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.890142 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.890260 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.896379 (+ 6119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:01.896386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:01.896391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:01.896435 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:01.900548 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.901412 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.901418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.901421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.901568 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:01.901993 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:01.901998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:01.902001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:01.902363 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:01.902622 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.903415 (+ 793us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.903579 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14163,"thread_start_us":17434,"threads_started":1}
I20260807 04:23:01.900569 8290 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:23:01.910539 8290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:01.946239 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.931330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42793 (local address 127.0.0.1:53056)
0807 04:23:01.932629 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.932650 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.932805 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.935179 (+ 2374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.935188 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.935229 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:01.935792 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.935809 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.938890 (+ 3081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.938901 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.942385 (+ 3484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.942424 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.945250 (+ 2826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.945259 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.945555 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.945593 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.945693 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.945858 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"mutex_wait_us":29,"spinlock_wait_cycles":11136,"thread_start_us":1365,"threads_started":1}
I20260807 04:23:01.946228 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.931590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.1.244.194:42793)
0807 04:23:01.934423 (+ 2833us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.934435 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.934471 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:23:01.934592 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.934603 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.934901 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.936019 (+ 1118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:01.936285 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.938435 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.943029 (+ 4594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.945013 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.945194 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.945453 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.945757 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.945964 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2588,"thread_start_us":2459,"threads_started":1}
I20260807 04:23:01.950521 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.931291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.1.244.193:44963)
0807 04:23:01.932163 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.932176 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.932958 (+ 782us) server_negotiation.cc:407] Connection header received
0807 04:23:01.933107 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.933119 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.933412 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.933636 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:01.940758 (+ 7122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.942758 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.948245 (+ 5487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.949594 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.949686 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.949824 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.950173 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.950298 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":521,"threads_started":1}
I20260807 04:23:01.950659 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.930963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44963 (local address 127.0.0.1:44404)
0807 04:23:01.932629 (+ 1666us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.932650 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.932805 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.939296 (+ 6491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.939301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.939330 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:01.939813 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.939828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.942954 (+ 3126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.942963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.948069 (+ 5106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.948089 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.949776 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.949783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.949986 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.950022 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.950115 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.950245 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1489,"spinlock_wait_cycles":7808,"thread_start_us":1253,"threads_started":1}
I20260807 04:23:01.951064 9361 tablet_service.cc:1467] Processing CreateTablet for tablet dc9505c6516643b18e8a650d5f23ac8b (DEFAULT_TABLE table=test-table [id=2851ea58ff9c431da07d1d632cc49b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:01.952399 9361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9505c6516643b18e8a650d5f23ac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:01.955454 9212 tablet_service.cc:1467] Processing CreateTablet for tablet dc9505c6516643b18e8a650d5f23ac8b (DEFAULT_TABLE table=test-table [id=2851ea58ff9c431da07d1d632cc49b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:01.956461 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.933119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35953 (local address 127.0.0.1:56788)
0807 04:23:01.945706 (+ 12587us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:01.945725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:01.945840 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:01.946781 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:01.946788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:01.946823 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:01.947302 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.947315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.952663 (+ 5348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.952667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.953901 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:01.953915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.955352 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.955359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:01.955570 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.955603 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:23:01.955691 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 04:23:01.955814 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12440,"thread_start_us":12283,"threads_started":1}
I20260807 04:23:01.956830 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:01.933413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.1.244.195:35953)
0807 04:23:01.945046 (+ 11633us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:01.945056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:01.945829 (+ 773us) server_negotiation.cc:407] Connection header received
0807 04:23:01.945978 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:01.945986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:01.946191 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:01.946772 (+ 581us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:01.947555 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.949163 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.954054 (+ 4891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:01.955151 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:01.955294 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:01.955491 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:01.956239 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 04:23:01.956510 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11440,"thread_start_us":11642,"threads_started":1}
I20260807 04:23:01.957265 9212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9505c6516643b18e8a650d5f23ac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:01.959658 9507 tablet_service.cc:1467] Processing CreateTablet for tablet dc9505c6516643b18e8a650d5f23ac8b (DEFAULT_TABLE table=test-table [id=2851ea58ff9c431da07d1d632cc49b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:01.960857 9507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9505c6516643b18e8a650d5f23ac8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:01.971122 9610 tablet_bootstrap.cc:492] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Bootstrap starting.
I20260807 04:23:01.973647 9612 tablet_bootstrap.cc:492] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Bootstrap starting.
I20260807 04:23:01.976717 9611 tablet_bootstrap.cc:492] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Bootstrap starting.
I20260807 04:23:01.976854 9612 tablet_bootstrap.cc:654] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:01.976794 9610 tablet_bootstrap.cc:654] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:01.978025 9610 log.cc:824] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:01.978493 9612 log.cc:824] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:01.980602 9611 tablet_bootstrap.cc:654] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:01.981212 9612 tablet_bootstrap.cc:492] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: No bootstrap required, opened a new log
I20260807 04:23:01.981391 9610 tablet_bootstrap.cc:492] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: No bootstrap required, opened a new log
I20260807 04:23:01.981523 9612 ts_tablet_manager.cc:1397] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:23:01.981702 9610 ts_tablet_manager.cc:1397] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 04:23:01.981981 9611 log.cc:824] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:01.985626 9611 tablet_bootstrap.cc:492] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: No bootstrap required, opened a new log
I20260807 04:23:01.985962 9611 ts_tablet_manager.cc:1397] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:23:01.989600 9612 raft_consensus.cc:348] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:01.990031 9612 raft_consensus.cc:374] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:01.990159 9612 raft_consensus.cc:729] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c676684f55c341c683428b058cb45faf, State: Initialized, Role: FOLLOWER
I20260807 04:23:01.990751 9612 consensus_queue.cc:260] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:01.992849 9612 ts_tablet_manager.cc:1428] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 04:23:01.993345 9610 raft_consensus.cc:348] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:01.994091 9610 raft_consensus.cc:374] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:01.994182 9610 raft_consensus.cc:729] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8f0386037d4c9aa7e6d041a2de9c57, State: Initialized, Role: FOLLOWER
I20260807 04:23:01.994761 9610 consensus_queue.cc:260] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:01.998517 9611 raft_consensus.cc:348] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:01.999017 9611 raft_consensus.cc:374] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:01.999128 9611 raft_consensus.cc:729] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4e9fd7229f4f43a80e8541c4c0c7b7, State: Initialized, Role: FOLLOWER
I20260807 04:23:01.999938 9611 consensus_queue.cc:260] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:02.000846 9429 heartbeater.cc:503] Master 127.1.244.254:39343 was elected leader, sending a full tablet report...
I20260807 04:23:02.004576 9610 ts_tablet_manager.cc:1428] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Time spent starting tablet: real 0.023s user 0.017s sys 0.004s
I20260807 04:23:02.005266 9611 ts_tablet_manager.cc:1428] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Time spent starting tablet: real 0.019s user 0.012s sys 0.008s
I20260807 04:23:02.005800 9574 heartbeater.cc:503] Master 127.1.244.254:39343 was elected leader, sending a full tablet report...
W20260807 04:23:02.061420 9576 tablet.cc:2369] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:23:02.080255 9280 tablet.cc:2369] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:23:02.107280 9616 raft_consensus.cc:482] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:02.107620 9616 raft_consensus.cc:504] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:02.109290 9616 leader_election.cc:288] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793), de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953)
I20260807 04:23:02.116824 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.110776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:58981 (local address 127.1.244.195:35953)
0807 04:23:02.110990 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.110996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.111274 (+ 278us) server_negotiation.cc:407] Connection header received
0807 04:23:02.111435 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.111440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.111524 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.111625 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:02.112343 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.113188 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.115016 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.115818 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.116022 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.116127 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.116442 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.116583 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:23:02.116930 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.110511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35953 (local address 127.1.244.193:58981)
0807 04:23:02.111174 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:02.111189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:02.111296 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:02.111647 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:02.111652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:02.111734 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:02.112143 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:02.112155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.113338 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.113342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:02.114861 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:02.114873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.116051 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.116059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:02.116283 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.116307 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:23:02.116382 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 04:23:02.116465 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":22,"thread_start_us":382,"threads_started":1}
I20260807 04:23:02.117051 9619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.109677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42793 (local address 127.1.244.193:35417)
0807 04:23:02.110391 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:02.110407 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:02.110605 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:02.111110 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:02.111116 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:02.111148 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:02.111572 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:02.111589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.113203 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.113210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:02.114967 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:02.114984 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.116211 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.116216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:02.116406 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.116437 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:23:02.116530 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:23:02.116645 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":25,"thread_start_us":366,"threads_started":1}
I20260807 04:23:02.117323 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.109839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35417 (local address 127.1.244.194:42793)
0807 04:23:02.110104 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.110112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.110522 (+ 410us) server_negotiation.cc:407] Connection header received
0807 04:23:02.110765 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.110773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.110867 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.110967 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:02.111797 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.113039 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.115140 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.115976 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.116127 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.116198 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.116583 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.116716 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:23:02.118115 9527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9505c6516643b18e8a650d5f23ac8b" candidate_uuid: "c676684f55c341c683428b058cb45faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" is_pre_election: true
I20260807 04:23:02.118299 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9505c6516643b18e8a650d5f23ac8b" candidate_uuid: "c676684f55c341c683428b058cb45faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" is_pre_election: true
I20260807 04:23:02.118746 9527 raft_consensus.cc:2454] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c676684f55c341c683428b058cb45faf in term 0.
I20260807 04:23:02.118836 9381 raft_consensus.cc:2454] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c676684f55c341c683428b058cb45faf in term 0.
I20260807 04:23:02.119454 9164 leader_election.cc:302] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c676684f55c341c683428b058cb45faf, de4e9fd7229f4f43a80e8541c4c0c7b7; no voters:
I20260807 04:23:02.119858 9616 raft_consensus.cc:2781] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:02.120069 9616 raft_consensus.cc:482] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:02.120201 9616 raft_consensus.cc:3037] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:02.122521 9616 raft_consensus.cc:504] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:02.123178 9616 leader_election.cc:288] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [CANDIDATE]: Term 1 election: Requested vote from peers 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793), de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953)
I20260807 04:23:02.123668 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9505c6516643b18e8a650d5f23ac8b" candidate_uuid: "c676684f55c341c683428b058cb45faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57"
I20260807 04:23:02.123704 9527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9505c6516643b18e8a650d5f23ac8b" candidate_uuid: "c676684f55c341c683428b058cb45faf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7"
I20260807 04:23:02.123871 9381 raft_consensus.cc:3037] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:02.123955 9527 raft_consensus.cc:3037] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:02.126188 9381 raft_consensus.cc:2454] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c676684f55c341c683428b058cb45faf in term 1.
I20260807 04:23:02.126211 9527 raft_consensus.cc:2454] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c676684f55c341c683428b058cb45faf in term 1.
I20260807 04:23:02.126551 9164 leader_election.cc:302] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b8f0386037d4c9aa7e6d041a2de9c57, c676684f55c341c683428b058cb45faf; no voters:
I20260807 04:23:02.126818 9616 raft_consensus.cc:2781] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:02.127286 9616 raft_consensus.cc:686] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 1 LEADER]: Becoming Leader. State: Replica: c676684f55c341c683428b058cb45faf, State: Running, Role: LEADER
I20260807 04:23:02.127857 9616 consensus_queue.cc:237] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } }
I20260807 04:23:02.132942 8290 catalog_manager.cc:5495] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf reported cstate change: term changed from 0 to 1, leader changed from <none> to c676684f55c341c683428b058cb45faf (127.1.244.193). New cstate: current_term: 1 leader_uuid: "c676684f55c341c683428b058cb45faf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } health_report { overall_health: UNKNOWN } } }
I20260807 04:23:02.183663 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.177604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.1.244.193:44963)
0807 04:23:02.177802 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.177809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.177971 (+ 162us) server_negotiation.cc:407] Connection header received
0807 04:23:02.178230 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.178234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.178312 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.178467 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:02.178851 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.179554 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.180988 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.181982 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.182114 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.182685 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:02.183142 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:02.183249 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.183329 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.183391 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:23:02.192196 9381 raft_consensus.cc:1261] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Refusing update from remote peer c676684f55c341c683428b058cb45faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:02.192231 9527 raft_consensus.cc:1261] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Refusing update from remote peer c676684f55c341c683428b058cb45faf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:02.193234 9621 consensus_queue.cc:1035] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:02.193574 9616 consensus_queue.cc:1035] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:23:02.206990 9437 tablet.cc:2369] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:23:02.210563 9624 mvcc.cc:205] Tried to move back new op lower bound from 7315769680648003584 to 7315769680399396864. Current Snapshot: MvccSnapshot[applied={T|T < 7315769680648003584}]
I20260807 04:23:02.216914 9625 mvcc.cc:205] Tried to move back new op lower bound from 7315769680648003584 to 7315769680399396864. Current Snapshot: MvccSnapshot[applied={T|T < 7315769680648003584 or (T in {7315769680648003584})}]
I20260807 04:23:02.227249 8289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:02.227535 8289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:02.233798 8289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f454652f335495597b168618f546d2a: Sending DeleteTablet for 3 replicas of tablet dc9505c6516643b18e8a650d5f23ac8b
I20260807 04:23:02.234961 9361 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9505c6516643b18e8a650d5f23ac8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:02 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:53056
I20260807 04:23:02.235881 9212 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9505c6516643b18e8a650d5f23ac8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:02 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44404
I20260807 04:23:02.236258 9634 tablet_replica.cc:330] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: stopping tablet replica
I20260807 04:23:02.236634 9507 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9505c6516643b18e8a650d5f23ac8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:02 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:56788
I20260807 04:23:02.236901 9634 raft_consensus.cc:2229] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:02.237607 9634 raft_consensus.cc:2258] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:02.240656 9634 ts_tablet_manager.cc:1905] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:02.248446 9636 tablet_replica.cc:330] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: stopping tablet replica
I20260807 04:23:02.248983 9636 raft_consensus.cc:2229] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:02.249547 9636 raft_consensus.cc:2258] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:02.251737 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8257
I20260807 04:23:02.251840 9635 tablet_replica.cc:330] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: stopping tablet replica
I20260807 04:23:02.252252 9634 ts_tablet_manager.cc:1918] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:02.252328 9635 raft_consensus.cc:2229] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:23:02.252543 9634 log.cc:1196] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/dc9505c6516643b18e8a650d5f23ac8b
I20260807 04:23:02.252921 8320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:02.252995 9635 raft_consensus.cc:2258] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:02.253584 9634 ts_tablet_manager.cc:1939] T dc9505c6516643b18e8a650d5f23ac8b P 7b8f0386037d4c9aa7e6d041a2de9c57: Deleting consensus metadata
I20260807 04:23:02.255016 9635 ts_tablet_manager.cc:1905] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:02.255762 9636 ts_tablet_manager.cc:1905] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:02.262120 9635 ts_tablet_manager.cc:1918] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:02.262292 9635 log.cc:1196] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/dc9505c6516643b18e8a650d5f23ac8b
I20260807 04:23:02.262897 9635 ts_tablet_manager.cc:1939] T dc9505c6516643b18e8a650d5f23ac8b P c676684f55c341c683428b058cb45faf: Deleting consensus metadata
I20260807 04:23:02.264851 9636 ts_tablet_manager.cc:1918] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:02.265076 9636 log.cc:1196] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/dc9505c6516643b18e8a650d5f23ac8b
I20260807 04:23:02.265686 9636 ts_tablet_manager.cc:1939] T dc9505c6516643b18e8a650d5f23ac8b P de4e9fd7229f4f43a80e8541c4c0c7b7: Deleting consensus metadata
I20260807 04:23:02.276288 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.243193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.1.244.253:38199)
0807 04:23:02.243429 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.243439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.250931 (+ 7492us) server_negotiation.cc:407] Connection header received
0807 04:23:02.251020 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.251028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.251132 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.251246 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:02.254199 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.255244 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.263619 (+ 8375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.264458 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.264569 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.268438 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:02.268448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:02.268455 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:02.268512 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:02.273264 (+ 4752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.274127 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.274132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.274142 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.274271 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.274676 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.274682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.274688 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.275049 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:02.275312 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.275685 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.275934 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 04:23:02.293591 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.250809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.1.244.252:43429)
0807 04:23:02.251617 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.251624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.276760 (+ 25136us) server_negotiation.cc:407] Connection header received
0807 04:23:02.277029 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.277035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.277122 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.277195 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:02.279305 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.280505 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.282322 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.282981 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.283114 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.286428 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:02.286435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:02.286441 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:02.286486 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:02.290301 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.291212 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.291218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.291221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.291363 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.291820 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.291825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.291828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.292215 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:02.292486 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.292840 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.293069 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629}
I20260807 04:23:02.397567 8275 catalog_manager.cc:4841] TS 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793): tablet dc9505c6516643b18e8a650d5f23ac8b (table test-table [id=2851ea58ff9c431da07d1d632cc49b08]) successfully deleted
W20260807 04:23:02.397967 8275 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.1.244.194:42793, user_credentials={real_user=slave}} blocked reactor thread for 141092us
I20260807 04:23:02.397545 8276 catalog_manager.cc:4841] TS c676684f55c341c683428b058cb45faf (127.1.244.193:44963): tablet dc9505c6516643b18e8a650d5f23ac8b (table test-table [id=2851ea58ff9c431da07d1d632cc49b08]) successfully deleted
I20260807 04:23:02.398454 8275 catalog_manager.cc:4841] TS de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953): tablet dc9505c6516643b18e8a650d5f23ac8b (table test-table [id=2851ea58ff9c431da07d1d632cc49b08]) successfully deleted
W20260807 04:23:02.398553 8276 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.1.244.193:44963, user_credentials={real_user=slave}} blocked reactor thread for 133682us
I20260807 04:23:02.399072 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8257
I20260807 04:23:02.446676 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.429628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.1.244.253:38199)
0807 04:23:02.429865 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.429874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.430821 (+ 947us) server_negotiation.cc:407] Connection header received
0807 04:23:02.431342 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.431347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.431418 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.431493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:02.433896 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.434925 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.436296 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.436782 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.436891 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.440382 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:02.440387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:02.440390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:02.440420 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:02.443594 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.444379 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.444383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.444386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.444570 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.445066 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.445071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.445075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.445405 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:02.445680 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.446052 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.446293 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 04:23:02.448210 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:02.428692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.1.244.252:43429)
0807 04:23:02.428929 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:02.428937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:02.430544 (+ 1607us) server_negotiation.cc:407] Connection header received
0807 04:23:02.430953 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:02.430962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:02.431071 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:02.431152 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:02.433545 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.434669 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.436685 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:02.437275 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:02.437421 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:02.441043 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:02.441051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:02.441058 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:02.441102 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:02.444338 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.445109 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.445116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.445122 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.445277 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:02.445721 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:02.445726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:02.445730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:02.446111 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:02.446509 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:02.447603 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0807 04:23:02.447843 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:23:03.212710 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.211802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:47409)
0807 04:23:03.212390 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.212518 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":300,"threads_started":1}
W20260807 04:23:03.213290 9429 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:39343 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.219422 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.218595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:48589)
0807 04:23:03.219117 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.219252 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":299,"threads_started":1}
W20260807 04:23:03.219836 9574 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:39343 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.224527 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.223825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:36967)
0807 04:23:03.224245 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.224376 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":232,"threads_started":1}
W20260807 04:23:03.224907 9278 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:39343 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.734725 9649 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0f454652f335495597b168618f546d2a)
I20260807 04:23:03.734968 9649 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:03.735212 9650 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0f454652f335495597b168618f546d2a)
I20260807 04:23:03.735508 9650 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:03.735870 9649 leader_election.cc:288] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343), bd6b742437f84714b619c787d28a1df3 (127.1.244.252:43429)
I20260807 04:23:03.737311 9650 leader_election.cc:288] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343), d47bccfe2ef84e068685777520783522 (127.1.244.253:38199)
I20260807 04:23:03.738396 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.736078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:37996)
0807 04:23:03.737930 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.738122 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1714,"threads_started":1}
I20260807 04:23:03.738785 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.737362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38000)
0807 04:23:03.738207 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.738471 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":644,"threads_started":1}
W20260807 04:23:03.739621 9078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
W20260807 04:23:03.740298 8883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.748370 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.736102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43429 (local address 127.0.0.1:36626)
0807 04:23:03.738631 (+ 2529us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.738649 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:03.738784 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:03.739665 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:03.739671 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:03.739705 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:03.740123 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:03.740141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.742365 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.742374 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:03.745195 (+ 2821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:03.745218 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.747179 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.747188 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:03.747465 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:03.747525 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 04:23:03.747619 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 04:23:03.747817 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2385,"mutex_wait_us":1,"spinlock_wait_cycles":101120,"thread_start_us":1577,"threads_started":1}
I20260807 04:23:03.748678 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.739302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:38199 (local address 127.0.0.1:59836)
0807 04:23:03.739453 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.739471 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:03.739600 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:03.739999 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:03.740003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:03.740031 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:03.740296 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:03.740311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.742595 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.742604 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:03.745354 (+ 2750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:03.745373 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.747609 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.747618 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:03.747941 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:03.748002 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 04:23:03.748085 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 04:23:03.748281 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 04:23:03.749279 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.746955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38002)
0807 04:23:03.748946 (+ 1991us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.749055 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1857,"threads_started":1}
I20260807 04:23:03.750257 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.736428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.1.244.252:43429)
0807 04:23:03.739301 (+ 2873us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:03.739309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:03.739335 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:23:03.739436 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:03.739441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:03.739531 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:03.739647 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:03.740382 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.742150 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.745453 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.746948 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.749249 (+ 2301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:03.749520 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:03.749587 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 04:23:03.749776 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2747,"thread_start_us":2599,"threads_started":1}
I20260807 04:23:03.750381 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.738528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.1.244.253:38199)
0807 04:23:03.739282 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:03.739287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:03.739563 (+ 276us) server_negotiation.cc:407] Connection header received
0807 04:23:03.739755 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:03.739763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:03.739845 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:03.740446 (+ 601us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:03.740692 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.742382 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.745625 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:03.747363 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:03.749500 (+ 2137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:03.749669 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:03.749733 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 04:23:03.749923 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":437,"threads_started":1}
W20260807 04:23:03.751209 9078 leader_election.cc:334] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.751245 9104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d47bccfe2ef84e068685777520783522" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd6b742437f84714b619c787d28a1df3" is_pre_election: true
I20260807 04:23:03.751648 9104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d47bccfe2ef84e068685777520783522 in term 1.
I20260807 04:23:03.751982 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.751519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38018)
0807 04:23:03.751648 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.751735 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 04:23:03.751967 8910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd6b742437f84714b619c787d28a1df3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d47bccfe2ef84e068685777520783522" is_pre_election: true
W20260807 04:23:03.752171 8883 leader_election.cc:334] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.752318 8910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b742437f84714b619c787d28a1df3 in term 1.
I20260807 04:23:03.752418 8883 leader_election.cc:302] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6b742437f84714b619c787d28a1df3, d47bccfe2ef84e068685777520783522; no voters: 0f454652f335495597b168618f546d2a
I20260807 04:23:03.752722 9649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 04:23:03.752846 9649 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0f454652f335495597b168618f546d2a)
I20260807 04:23:03.752833 9078 leader_election.cc:302] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6b742437f84714b619c787d28a1df3, d47bccfe2ef84e068685777520783522; no voters: 0f454652f335495597b168618f546d2a
I20260807 04:23:03.752960 9649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:23:03.753175 9650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 04:23:03.753310 9650 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0f454652f335495597b168618f546d2a)
I20260807 04:23:03.753435 9650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:23:03.755183 9649 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:03.755506 9650 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:03.755805 9649 leader_election.cc:288] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 2 election: Requested vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343), bd6b742437f84714b619c787d28a1df3 (127.1.244.252:43429)
I20260807 04:23:03.756234 9650 leader_election.cc:288] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 2 election: Requested vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343), d47bccfe2ef84e068685777520783522 (127.1.244.253:38199)
I20260807 04:23:03.756325 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.755883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38022)
0807 04:23:03.756115 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.756197 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 04:23:03.756415 9104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d47bccfe2ef84e068685777520783522" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd6b742437f84714b619c787d28a1df3"
I20260807 04:23:03.756664 9104 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d47bccfe2ef84e068685777520783522 in current term 2: Already voted for candidate bd6b742437f84714b619c787d28a1df3 in this term.
I20260807 04:23:03.757335 9652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.756947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38032)
0807 04:23:03.757136 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.757221 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 04:23:03.757513 8910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd6b742437f84714b619c787d28a1df3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d47bccfe2ef84e068685777520783522"
I20260807 04:23:03.757704 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.756941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38030)
0807 04:23:03.757278 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.757484 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260807 04:23:03.757776 8910 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bd6b742437f84714b619c787d28a1df3 in current term 2: Already voted for candidate d47bccfe2ef84e068685777520783522 in this term.
W20260807 04:23:03.758080 8883 leader_election.cc:334] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.758222 8883 leader_election.cc:302] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d47bccfe2ef84e068685777520783522; no voters: 0f454652f335495597b168618f546d2a, bd6b742437f84714b619c787d28a1df3
I20260807 04:23:03.758520 9649 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 04:23:03.758827 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:03.758458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38046)
0807 04:23:03.758619 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:03.758708 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 04:23:03.758998 9078 leader_election.cc:334] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:03.759164 9078 leader_election.cc:302] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd6b742437f84714b619c787d28a1df3; no voters: 0f454652f335495597b168618f546d2a, d47bccfe2ef84e068685777520783522
I20260807 04:23:03.759447 9650 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 04:23:04.214783 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:04.214073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:54553)
0807 04:23:04.214516 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:04.214603 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":240,"threads_started":1}
I20260807 04:23:04.221957 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:04.220609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:55103)
0807 04:23:04.221065 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:04.221168 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":229,"threads_started":1}
I20260807 04:23:04.226082 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:04.225455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:33231)
0807 04:23:04.225824 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:04.225901 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":189,"threads_started":1}
I20260807 04:23:05.216495 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.215665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:48735)
0807 04:23:05.216220 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.216313 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":292,"threads_started":1}
I20260807 04:23:05.223850 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.222984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:43463)
0807 04:23:05.223482 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.223589 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":289,"threads_started":1}
I20260807 04:23:05.227440 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.226821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:39925)
0807 04:23:05.227182 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.227271 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":193,"threads_started":1}
I20260807 04:23:05.408249 9665 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:05.408483 9665 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:05.409035 9665 leader_election.cc:288] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343), d47bccfe2ef84e068685777520783522 (127.1.244.253:38199)
I20260807 04:23:05.410054 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.409246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38286)
0807 04:23:05.409762 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.409890 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":264,"threads_started":1}
I20260807 04:23:05.410262 8910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd6b742437f84714b619c787d28a1df3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d47bccfe2ef84e068685777520783522" is_pre_election: true
I20260807 04:23:05.410533 8910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd6b742437f84714b619c787d28a1df3 in term 2.
I20260807 04:23:05.410990 9078 leader_election.cc:302] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd6b742437f84714b619c787d28a1df3, d47bccfe2ef84e068685777520783522; no voters:
I20260807 04:23:05.411266 9665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 04:23:05.411394 9665 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:05.411528 9665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 2 FOLLOWER]: Advancing to term 3
I20260807 04:23:05.411751 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.411393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38292)
0807 04:23:05.411528 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.411615 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 04:23:05.412025 9078 leader_election.cc:334] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:05.414001 9665 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:05.414521 9665 leader_election.cc:288] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 3 election: Requested vote from peers 0f454652f335495597b168618f546d2a (127.1.244.254:39343), d47bccfe2ef84e068685777520783522 (127.1.244.253:38199)
I20260807 04:23:05.415083 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.414699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38294)
0807 04:23:05.414839 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.414941 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 04:23:05.415266 8910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd6b742437f84714b619c787d28a1df3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d47bccfe2ef84e068685777520783522"
I20260807 04:23:05.415504 8910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 2 FOLLOWER]: Advancing to term 3
I20260807 04:23:05.416139 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.415780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38298)
0807 04:23:05.415968 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.416031 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 04:23:05.416369 9078 leader_election.cc:334] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
I20260807 04:23:05.417802 8910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd6b742437f84714b619c787d28a1df3 in term 3.
I20260807 04:23:05.418208 9078 leader_election.cc:302] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd6b742437f84714b619c787d28a1df3, d47bccfe2ef84e068685777520783522; no voters: 0f454652f335495597b168618f546d2a
I20260807 04:23:05.418475 9665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 04:23:05.418913 9665 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [term 3 LEADER]: Becoming Leader. State: Replica: bd6b742437f84714b619c787d28a1df3, State: Running, Role: LEADER
I20260807 04:23:05.419283 9665 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } }
I20260807 04:23:05.421447 9668 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd6b742437f84714b619c787d28a1df3. Latest consensus state: current_term: 3 leader_uuid: "bd6b742437f84714b619c787d28a1df3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:05.421766 9668 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:05.422429 9670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:05.426182 9670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:05.426954 9670 catalog_manager.cc:1259] Loaded cluster ID: fd18d5a3c3a34e82b9af0a0b9bbc26d1
I20260807 04:23:05.427023 9670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:05.428243 9670 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:05.429028 9670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Loaded TSK: 0
I20260807 04:23:05.429837 9670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:23:05.479091 9094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:23:05.486132 9094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:05.490198 8910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [term 3 FOLLOWER]: Refusing update from remote peer bd6b742437f84714b619c787d28a1df3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 04:23:05.490290 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.489724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38322)
0807 04:23:05.490021 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.490110 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260807 04:23:05.490993 9668 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 04:23:05.491537 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.491142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38324)
0807 04:23:05.491324 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.491432 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 04:23:05.491829 9078 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 -> Peer 0f454652f335495597b168618f546d2a (127.1.244.254:39343): Couldn't send request to peer 0f454652f335495597b168618f546d2a. Status: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 04:23:05.494059 9673 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd6b742437f84714b619c787d28a1df3. Latest consensus state: current_term: 3 leader_uuid: "bd6b742437f84714b619c787d28a1df3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:05.494416 9673 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:23:05.495482 9668 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bd6b742437f84714b619c787d28a1df3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:05.495791 9668 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3 [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:05.496050 9673 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bd6b742437f84714b619c787d28a1df3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0f454652f335495597b168618f546d2a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39343 } } peers { permanent_uuid: "d47bccfe2ef84e068685777520783522" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 38199 } attrs { promote: false } } peers { permanent_uuid: "bd6b742437f84714b619c787d28a1df3" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43429 } attrs { promote: false } } }
I20260807 04:23:05.496301 9673 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47bccfe2ef84e068685777520783522 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:23:05.496469 9674 mvcc.cc:205] Tried to move back new op lower bound from 7315769694161129472 to 7315769693881323520. Current Snapshot: MvccSnapshot[applied={T|T < 7315769694161129472}]
I20260807 04:23:05.514441 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.504518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:42793 (local address 127.0.0.1:53072)
0807 04:23:05.504748 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.504775 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:05.504906 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:05.505787 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:05.505795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:05.505832 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:05.506284 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.506299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.508321 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.508330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.511097 (+ 2767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.511116 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.513381 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.513388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.513638 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.513712 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 04:23:05.513795 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 04:23:05.513992 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"spinlock_wait_cycles":14592}
I20260807 04:23:05.515825 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.505041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35953 (local address 127.0.0.1:56800)
0807 04:23:05.505740 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.505761 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:05.505892 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:05.509565 (+ 3673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:05.509570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:05.509597 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:05.510006 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.510015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.511181 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.511188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.513369 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.513385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.514896 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.514918 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.515141 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.515180 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 04:23:05.515262 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 04:23:05.515416 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1,"thread_start_us":1751,"threads_started":1}
I20260807 04:23:05.516537 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.504644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.1.244.194:42793)
0807 04:23:05.505410 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.505417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.505435 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:23:05.505510 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.505515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.505606 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.506465 (+ 859us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:05.506706 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.508081 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.511351 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.513115 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.515120 (+ 2005us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.515775 (+ 655us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.515838 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.516071 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":493,"threads_started":1}
I20260807 04:23:05.517305 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.505538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.1.244.195:35953)
0807 04:23:05.509265 (+ 3727us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.509270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.509287 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:23:05.509348 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.509353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.509422 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.509551 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:05.510214 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.511034 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.513640 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.514505 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.516430 (+ 1925us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.516553 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.516607 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.516952 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3614,"thread_start_us":3489,"threads_started":1}
I20260807 04:23:05.517428 9361 tablet_service.cc:1467] Processing CreateTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 (DEFAULT_TABLE table=test-table [id=7b81b0e195d34c0a835ba72ba87f136d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:05.517916 9361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bae6b04ace54136b95b3ea3a283f9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:05.518036 9507 tablet_service.cc:1467] Processing CreateTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 (DEFAULT_TABLE table=test-table [id=7b81b0e195d34c0a835ba72ba87f136d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:05.518497 9507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bae6b04ace54136b95b3ea3a283f9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:05.521966 9666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.505512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44963 (local address 127.0.0.1:44424)
0807 04:23:05.514731 (+ 9219us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.514752 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:05.514883 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:05.515285 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:05.515292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:05.515325 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:05.515571 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.515586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.517580 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.517586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.519394 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.519411 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.521149 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.521158 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.521343 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.521379 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:23:05.521469 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 04:23:05.521596 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9090,"threads_started":1}
I20260807 04:23:05.522290 9684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.505978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.1.244.193:44963)
0807 04:23:05.506687 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.506705 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.514868 (+ 8163us) server_negotiation.cc:407] Connection header received
0807 04:23:05.515066 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.515072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.515153 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.515261 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:05.516314 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.517437 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.519579 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.520987 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.521119 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.521226 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.521526 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.521685 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":735,"threads_started":1}
I20260807 04:23:05.524171 9685 tablet_bootstrap.cc:492] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Bootstrap starting.
I20260807 04:23:05.525473 9686 tablet_bootstrap.cc:492] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Bootstrap starting.
I20260807 04:23:05.526611 9685 tablet_bootstrap.cc:654] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:05.527118 9686 tablet_bootstrap.cc:654] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:05.528847 9212 tablet_service.cc:1467] Processing CreateTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 (DEFAULT_TABLE table=test-table [id=7b81b0e195d34c0a835ba72ba87f136d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:05.529330 9212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bae6b04ace54136b95b3ea3a283f9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:05.529904 9686 tablet_bootstrap.cc:492] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: No bootstrap required, opened a new log
I20260807 04:23:05.530081 9686 ts_tablet_manager.cc:1397] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 04:23:05.530900 9686 raft_consensus.cc:348] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.531179 9686 raft_consensus.cc:374] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:05.531287 9686 raft_consensus.cc:729] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4e9fd7229f4f43a80e8541c4c0c7b7, State: Initialized, Role: FOLLOWER
I20260807 04:23:05.531560 9686 consensus_queue.cc:260] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.532518 9685 tablet_bootstrap.cc:492] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: No bootstrap required, opened a new log
I20260807 04:23:05.532703 9685 ts_tablet_manager.cc:1397] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260807 04:23:05.533311 9685 raft_consensus.cc:348] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.533529 9685 raft_consensus.cc:374] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:05.533627 9685 raft_consensus.cc:729] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8f0386037d4c9aa7e6d041a2de9c57, State: Initialized, Role: FOLLOWER
I20260807 04:23:05.533921 9685 consensus_queue.cc:260] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.534768 9685 ts_tablet_manager.cc:1428] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 04:23:05.536278 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.535652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:57211)
0807 04:23:05.536037 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.536130 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273}
I20260807 04:23:05.538017 9686 ts_tablet_manager.cc:1428] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260807 04:23:05.538033 9428 heartbeater.cc:503] Master 127.1.244.252:43429 was elected leader, sending a full tablet report...
I20260807 04:23:05.539216 9572 heartbeater.cc:503] Master 127.1.244.252:43429 was elected leader, sending a full tablet report...
I20260807 04:23:05.540753 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.540331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:40073)
0807 04:23:05.540494 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.540604 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 04:23:05.542060 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.541656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:59709)
0807 04:23:05.541818 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.541882 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 04:23:05.544543 9692 tablet_bootstrap.cc:492] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Bootstrap starting.
I20260807 04:23:05.546303 9692 tablet_bootstrap.cc:654] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:05.548249 9692 tablet_bootstrap.cc:492] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: No bootstrap required, opened a new log
I20260807 04:23:05.548369 9692 ts_tablet_manager.cc:1397] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 04:23:05.548972 9692 raft_consensus.cc:348] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.549157 9692 raft_consensus.cc:374] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:05.549229 9692 raft_consensus.cc:729] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c676684f55c341c683428b058cb45faf, State: Initialized, Role: FOLLOWER
I20260807 04:23:05.549489 9692 consensus_queue.cc:260] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.550283 9692 ts_tablet_manager.cc:1428] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:23:05.550884 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.550502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:44505)
0807 04:23:05.550648 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.550732 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 04:23:05.552060 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.551514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:47663)
0807 04:23:05.551816 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.551890 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260807 04:23:05.552311 9277 heartbeater.cc:503] Master 127.1.244.252:43429 was elected leader, sending a full tablet report...
I20260807 04:23:05.711561 9691 raft_consensus.cc:482] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:05.711774 9691 raft_consensus.cc:504] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.713495 9691 leader_election.cc:288] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953), c676684f55c341c683428b058cb45faf (127.1.244.193:44963)
I20260807 04:23:05.720016 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.714052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43521 (local address 127.1.244.195:35953)
0807 04:23:05.714243 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.714252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.714274 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:23:05.714366 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.714369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.714437 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.714542 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:05.715084 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.716136 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.717950 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.718936 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.719055 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.719114 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.719602 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.719742 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 04:23:05.720036 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.713765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:35953 (local address 127.1.244.194:43521)
0807 04:23:05.714039 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.714060 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:05.714204 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:05.714574 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:05.714579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:05.714608 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:05.714930 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.714946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.716262 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.716268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.717792 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.717804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.719113 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.719120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.719336 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.719377 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 04:23:05.719556 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 04:23:05.719718 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":1}
I20260807 04:23:05.720502 9684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.713964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:38289 (local address 127.1.244.193:44963)
0807 04:23:05.714177 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.714184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.714850 (+ 666us) server_negotiation.cc:407] Connection header received
0807 04:23:05.714988 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.714996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.715079 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.715149 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:05.715697 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.716596 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.718476 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.719436 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.719587 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.719657 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.720082 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.720222 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:23:05.720626 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.714096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44963 (local address 127.1.244.194:38289)
0807 04:23:05.714738 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.714752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:05.714860 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:05.715200 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:05.715207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:05.715234 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:05.715558 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.715570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.716722 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.716725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.718303 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:05.718320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.719686 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.719693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:05.719870 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.719936 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 04:23:05.720051 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 04:23:05.720207 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":42,"thread_start_us":394,"threads_started":1}
I20260807 04:23:05.720969 9527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bae6b04ace54136b95b3ea3a283f9b5" candidate_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" is_pre_election: true
I20260807 04:23:05.721251 9527 raft_consensus.cc:2454] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b8f0386037d4c9aa7e6d041a2de9c57 in term 0.
I20260807 04:23:05.721637 9232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bae6b04ace54136b95b3ea3a283f9b5" candidate_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c676684f55c341c683428b058cb45faf" is_pre_election: true
I20260807 04:23:05.721844 9314 leader_election.cc:302] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b8f0386037d4c9aa7e6d041a2de9c57, de4e9fd7229f4f43a80e8541c4c0c7b7; no voters:
I20260807 04:23:05.722055 9232 raft_consensus.cc:2454] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b8f0386037d4c9aa7e6d041a2de9c57 in term 0.
I20260807 04:23:05.722237 9691 raft_consensus.cc:2781] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:05.722376 9691 raft_consensus.cc:482] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:05.722488 9691 raft_consensus.cc:3037] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:05.724745 9691 raft_consensus.cc:504] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.725274 9691 leader_election.cc:288] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [CANDIDATE]: Term 1 election: Requested vote from peers de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953), c676684f55c341c683428b058cb45faf (127.1.244.193:44963)
I20260807 04:23:05.725793 9527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bae6b04ace54136b95b3ea3a283f9b5" candidate_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7"
I20260807 04:23:05.725785 9232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bae6b04ace54136b95b3ea3a283f9b5" candidate_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c676684f55c341c683428b058cb45faf"
I20260807 04:23:05.725997 9527 raft_consensus.cc:3037] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:05.725997 9232 raft_consensus.cc:3037] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:05.729040 9527 raft_consensus.cc:2454] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b8f0386037d4c9aa7e6d041a2de9c57 in term 1.
I20260807 04:23:05.729040 9232 raft_consensus.cc:2454] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b8f0386037d4c9aa7e6d041a2de9c57 in term 1.
I20260807 04:23:05.729406 9315 leader_election.cc:302] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b8f0386037d4c9aa7e6d041a2de9c57, c676684f55c341c683428b058cb45faf; no voters:
I20260807 04:23:05.729684 9691 raft_consensus.cc:2781] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:05.730120 9691 raft_consensus.cc:686] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 LEADER]: Becoming Leader. State: Replica: 7b8f0386037d4c9aa7e6d041a2de9c57, State: Running, Role: LEADER
I20260807 04:23:05.731137 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.730624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:50625)
0807 04:23:05.730831 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.730935 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 04:23:05.730937 9691 consensus_queue.cc:237] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } }
I20260807 04:23:05.734790 9095 catalog_manager.cc:5495] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b8f0386037d4c9aa7e6d041a2de9c57" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 42793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 } health_report { overall_health: UNKNOWN } } }
I20260807 04:23:05.752554 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.747416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53078 (local address 127.1.244.194:42793)
0807 04:23:05.747612 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.747620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.747641 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:23:05.747881 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.747885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.748007 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.748151 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:05.748496 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.749182 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.750488 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.750897 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.751027 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.751438 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:05.751976 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:05.752121 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.752203 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.752280 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:23:05.756382 9527 raft_consensus.cc:1261] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Refusing update from remote peer 7b8f0386037d4c9aa7e6d041a2de9c57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:05.756646 9232 raft_consensus.cc:1261] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Refusing update from remote peer 7b8f0386037d4c9aa7e6d041a2de9c57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:05.757094 9696 consensus_queue.cc:1035] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de4e9fd7229f4f43a80e8541c4c0c7b7" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:05.757445 9691 consensus_queue.cc:1035] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c676684f55c341c683428b058cb45faf" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:05.759622 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.759169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:39675)
0807 04:23:05.759360 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.759477 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 04:23:05.763607 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.763134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:56427)
0807 04:23:05.763356 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.763457 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260807 04:23:05.765319 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.764752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:55667)
0807 04:23:05.765050 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.765143 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260807 04:23:05.768229 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.767669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.195:44491)
0807 04:23:05.767983 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.768065 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260807 04:23:05.776152 9095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:05.776510 9095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:05.780790 9695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.780278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.194:46317)
0807 04:23:05.780506 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.780601 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 04:23:05.782346 9664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.781873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.1.244.193:33645)
0807 04:23:05.782082 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.782169 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 04:23:05.787859 9095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd6b742437f84714b619c787d28a1df3: Sending DeleteTablet for 3 replicas of tablet 9bae6b04ace54136b95b3ea3a283f9b5
I20260807 04:23:05.789100 9507 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:05 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:56800
I20260807 04:23:05.789235 9506 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:05 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:56800
I20260807 04:23:05.789657 9361 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:05 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:53072
I20260807 04:23:05.790763 9212 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:05 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:44424
I20260807 04:23:05.791159 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bae6b04ace54136b95b3ea3a283f9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:05 UTC) from {username='oryx', principal='oryx/127.1.244.252@KRBTEST.COM'} at 127.0.0.1:44424
I20260807 04:23:05.791822 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 9066
I20260807 04:23:05.797636 9711 tablet_replica.cc:330] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: stopping tablet replica
I20260807 04:23:05.797696 9712 tablet_replica.cc:330] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: stopping tablet replica
I20260807 04:23:05.797906 9712 raft_consensus.cc:2229] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:05.797842 9711 raft_consensus.cc:2229] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:23:05.798511 9710 tablet_replica.cc:330] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: stopping tablet replica
I20260807 04:23:05.798599 9711 raft_consensus.cc:2258] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:05.798864 9712 raft_consensus.cc:2258] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:05.799237 9710 raft_consensus.cc:2229] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:05.799852 9711 ts_tablet_manager.cc:1905] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:05.800158 9710 raft_consensus.cc:2258] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:05.801357 9710 ts_tablet_manager.cc:1905] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:05.801468 9712 ts_tablet_manager.cc:1905] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:05.808393 9710 ts_tablet_manager.cc:1918] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:05.808604 9710 log.cc:1196] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/9bae6b04ace54136b95b3ea3a283f9b5
I20260807 04:23:05.809221 9710 ts_tablet_manager.cc:1939] T 9bae6b04ace54136b95b3ea3a283f9b5 P de4e9fd7229f4f43a80e8541c4c0c7b7: Deleting consensus metadata
I20260807 04:23:05.809938 9712 ts_tablet_manager.cc:1918] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:05.810133 9712 log.cc:1196] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/9bae6b04ace54136b95b3ea3a283f9b5
I20260807 04:23:05.810493 9080 catalog_manager.cc:4841] TS de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953): tablet 9bae6b04ace54136b95b3ea3a283f9b5 (table test-table [id=7b81b0e195d34c0a835ba72ba87f136d]) successfully deleted
I20260807 04:23:05.810702 9712 ts_tablet_manager.cc:1939] T 9bae6b04ace54136b95b3ea3a283f9b5 P c676684f55c341c683428b058cb45faf: Deleting consensus metadata
W20260807 04:23:05.811499 9080 catalog_manager.cc:4808] TS de4e9fd7229f4f43a80e8541c4c0c7b7 (127.1.244.195:35953): delete failed for tablet 9bae6b04ace54136b95b3ea3a283f9b5 because the tablet was not found. No further retry: Not found: Tablet not found: 9bae6b04ace54136b95b3ea3a283f9b5
I20260807 04:23:05.811947 9081 catalog_manager.cc:4841] TS c676684f55c341c683428b058cb45faf (127.1.244.193:44963): tablet 9bae6b04ace54136b95b3ea3a283f9b5 (table test-table [id=7b81b0e195d34c0a835ba72ba87f136d]) successfully deleted
W20260807 04:23:05.812762 9081 catalog_manager.cc:4808] TS c676684f55c341c683428b058cb45faf (127.1.244.193:44963): delete failed for tablet 9bae6b04ace54136b95b3ea3a283f9b5 because the tablet was not found. No further retry: Not found: Tablet not found: 9bae6b04ace54136b95b3ea3a283f9b5
I20260807 04:23:05.814247 9711 ts_tablet_manager.cc:1918] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:05.814374 9711 log.cc:1196] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/9bae6b04ace54136b95b3ea3a283f9b5
I20260807 04:23:05.814816 9711 ts_tablet_manager.cc:1939] T 9bae6b04ace54136b95b3ea3a283f9b5 P 7b8f0386037d4c9aa7e6d041a2de9c57: Deleting consensus metadata
I20260807 04:23:05.816071 9080 catalog_manager.cc:4841] TS 7b8f0386037d4c9aa7e6d041a2de9c57 (127.1.244.194:42793): tablet 9bae6b04ace54136b95b3ea3a283f9b5 (table test-table [id=7b81b0e195d34c0a835ba72ba87f136d]) successfully deleted
I20260807 04:23:05.823954 9684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.792596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.1.244.193:44963)
0807 04:23:05.792805 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.792813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.799423 (+ 6610us) server_negotiation.cc:407] Connection header received
0807 04:23:05.799727 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.799735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.799838 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.800343 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:05.805145 (+ 4802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.806329 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.813590 (+ 7261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.814036 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.814178 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.817219 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:05.817236 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:05.817257 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:05.817293 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:05.820642 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:05.821514 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:05.821525 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:05.821530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:05.821822 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:05.822312 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:05.822317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:05.822321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:05.822679 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:05.823006 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.823301 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.823524 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:23:05.824666 9272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:05.946667 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 9066
I20260807 04:23:05.980917 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 9289
I20260807 04:23:05.981909 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.981241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38328)
0807 04:23:05.981440 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.981626 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 04:23:05.983036 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.982674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38336)
0807 04:23:05.982824 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:05.982908 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 04:23:05.997573 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:05.981805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.1.244.194:42793)
0807 04:23:05.982065 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:05.982074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:05.982332 (+ 258us) server_negotiation.cc:407] Connection header received
0807 04:23:05.982623 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:05.982629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:05.982737 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:05.982843 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:05.984596 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.985771 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.987041 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:05.987652 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:05.987774 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:05.990299 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:05.990313 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:05.990330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:05.990367 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:05.994317 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:05.995108 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:05.995118 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:05.995122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:05.995294 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:05.995646 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:05.995651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:05.995655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:05.996171 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:05.996463 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:05.996925 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 04:23:05.997159 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 04:23:05.998328 9423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:06.118194 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 9289
I20260807 04:23:06.140273 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 9441
I20260807 04:23:06.153393 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:06.141036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.1.244.195:35953)
0807 04:23:06.141219 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:06.141227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:06.141320 (+ 93us) server_negotiation.cc:407] Connection header received
0807 04:23:06.141541 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:06.141547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:06.141636 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:06.141745 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:06.143094 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:06.143888 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:06.145266 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:06.145670 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:06.145764 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:06.148352 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:06.148367 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:06.148383 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:06.148418 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:06.151013 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:06.151601 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:06.151608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:06.151610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:06.151731 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:06.152100 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:06.152104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:06.152106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:06.152437 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:06.152665 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:06.152863 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 04:23:06.153028 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:23:06.154101 9567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:06.247332 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 9441
I20260807 04:23:06.267607 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8761
I20260807 04:23:06.268849 8931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:06.389459 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8761
I20260807 04:23:06.408447 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:06.408017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38352)
0807 04:23:06.408192 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:06.408282 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 04:23:06.409483 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:06.409148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39343 (local address 127.0.0.1:38354)
0807 04:23:06.409296 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:06.409359 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:39343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 04:23:06.412932 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 8863
I20260807 04:23:06.413882 9126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:06.536293 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 8863
2026-08-07T04:23:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11421 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:23:06 dist-test-slave-zkpd krb5kdc[9721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:23:06 dist-test-slave-zkpd krb5kdc[9721](info): set up 2 sockets
Aug 07 04:23:06 dist-test-slave-zkpd krb5kdc[9721](info): commencing operation
krb5kdc: starting...
W20260807 04:23:07.675470 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:23:07 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:23:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
Entry for principal oryx/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab.
I20260807 04:23:07.901312 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46539
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:08.228472 9734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:08.228765 9734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:08.228946 9734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:08.236135 9734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:08.236227 9734 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 04:23:08.236330 9734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:08.236434 9734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:08.236523 9734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:08.236627 9734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:08.246121 9734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46539
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.254.keytab
--principal=oryx/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:08.247309 9734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:08.248643 9734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:08.260847 9742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:08.260939 9740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:08.260952 9739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:08.261541 9734 server_base.cc:1034] running on GCE node
I20260807 04:23:08.262306 9734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:08.263588 9734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:08.264825 9734 hybrid_clock.cc:648] HybridClock initialized: now 1786076588264776 us; error 42 us; skew 500 ppm
I20260807 04:23:08.265149 9734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:08 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076588, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:08.268342 9734 init.cc:377] Logged in from keytab as oryx/127.1.244.254@KRBTEST.COM (short username oryx)
I20260807 04:23:08.269744 9734 webserver.cc:460] Webserver started at http://127.1.244.254:39267/ using document root <none> and password file <none>
I20260807 04:23:08.270419 9734 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:08.270555 9734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:08.270879 9734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:08.273823 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "fc7a009d61d149cfbbab2f0887497bde"
format_stamp: "Formatted at 2026-08-07 04:23:08 on dist-test-slave-zkpd"
server_key: "aa6218ef4a204c3cfd71a26737318fcd"
server_key_iv: "b0989b3bcd4957b826a9f9f8a5d0f739"
server_key_version: "encryptionkey@0"
I20260807 04:23:08.274633 9734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "fc7a009d61d149cfbbab2f0887497bde"
format_stamp: "Formatted at 2026-08-07 04:23:08 on dist-test-slave-zkpd"
server_key: "aa6218ef4a204c3cfd71a26737318fcd"
server_key_iv: "b0989b3bcd4957b826a9f9f8a5d0f739"
server_key_version: "encryptionkey@0"
I20260807 04:23:08.279522 9734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 04:23:08.282828 9749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:08.284165 9734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:08.284343 9734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "fc7a009d61d149cfbbab2f0887497bde"
format_stamp: "Formatted at 2026-08-07 04:23:08 on dist-test-slave-zkpd"
server_key: "aa6218ef4a204c3cfd71a26737318fcd"
server_key_iv: "b0989b3bcd4957b826a9f9f8a5d0f739"
server_key_version: "encryptionkey@0"
I20260807 04:23:08.284569 9734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:08.298745 9734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:08.302982 9734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:08.303238 9734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:08.324687 9734 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46539
I20260807 04:23:08.324719 9813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46539 every 8 connection(s)
I20260807 04:23:08.326069 9734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:23:08.327651 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 9734
I20260807 04:23:08.327867 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:23:08.328573 2003 external_mini_cluster.cc:1428] Setting key 804832c5600a6616d75b884d1d1ba5e7
I20260807 04:23:08.330248 9814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:23:08 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:08.344293 9814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Bootstrap starting.
I20260807 04:23:08.347585 9814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:08.348721 9814 log.cc:824] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:08.351372 9814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: No bootstrap required, opened a new log
I20260807 04:23:08.352773 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:08.330589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36194 (local address 127.1.244.254:46539)
0807 04:23:08.331397 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:08.331412 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:08.331469 (+ 57us) server_negotiation.cc:407] Connection header received
0807 04:23:08.333448 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:08.333473 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:08.334068 (+ 595us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:08.334507 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:08.335746 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.337833 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.339402 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.340517 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.340729 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:08.345530 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:08.345537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:08.345550 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:08.345586 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:08.348784 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:08.349498 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:08.349516 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:08.349522 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:08.349707 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:08.350110 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:08.350114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:08.350117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:08.350591 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:08.350897 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:08.351352 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 04:23:08.351759 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":322,"threads_started":1}
I20260807 04:23:08.359375 9814 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a009d61d149cfbbab2f0887497bde" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46539 } }
I20260807 04:23:08.359772 9814 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:08.359879 9814 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7a009d61d149cfbbab2f0887497bde, State: Initialized, Role: FOLLOWER
I20260807 04:23:08.360464 9814 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a009d61d149cfbbab2f0887497bde" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46539 } }
I20260807 04:23:08.360688 9814 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:23:08.360845 9814 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:23:08.361021 9814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:08.363023 9814 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a009d61d149cfbbab2f0887497bde" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46539 } }
I20260807 04:23:08.363493 9814 leader_election.cc:302] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc7a009d61d149cfbbab2f0887497bde; no voters:
I20260807 04:23:08.364095 9814 leader_election.cc:288] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:23:08.364228 9819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:08.365643 9819 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [term 1 LEADER]: Becoming Leader. State: Replica: fc7a009d61d149cfbbab2f0887497bde, State: Running, Role: LEADER
I20260807 04:23:08.366114 9819 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a009d61d149cfbbab2f0887497bde" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46539 } }
I20260807 04:23:08.366334 9814 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:08.369009 9820 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc7a009d61d149cfbbab2f0887497bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a009d61d149cfbbab2f0887497bde" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46539 } } }
I20260807 04:23:08.368948 9821 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc7a009d61d149cfbbab2f0887497bde. Latest consensus state: current_term: 1 leader_uuid: "fc7a009d61d149cfbbab2f0887497bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a009d61d149cfbbab2f0887497bde" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46539 } } }
I20260807 04:23:08.369412 9820 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:08.369418 9821 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:08.372195 9829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:08.376441 9829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:08.384388 9829 catalog_manager.cc:1347] Generated new cluster ID: a876aaeacfd74475a3bbea97dc43fc5b
I20260807 04:23:08.384505 9829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:08.405728 9829 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:23:08.406989 9829 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:08.415699 9829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Generated new TSK 0
I20260807 04:23:08.416621 9829 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
I20260807 04:23:08.489635 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46539
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:08.818964 9841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:08.819216 9841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:08.819314 9841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:08.826035 9841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:08.826126 9841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:08.826205 9841 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:23:08.826282 9841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:08.835480 9841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:46539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:08.836654 9841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:08.837883 9841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:08.849953 9846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:08.849947 9850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:08.849954 9847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:08.851001 9841 server_base.cc:1034] running on GCE node
I20260807 04:23:08.851471 9841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:08.852349 9841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:08.853572 9841 hybrid_clock.cc:648] HybridClock initialized: now 1786076588853522 us; error 25 us; skew 500 ppm
I20260807 04:23:08.853876 9841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:08 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076588, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:08.857221 9841 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260807 04:23:08.858687 9841 webserver.cc:460] Webserver started at http://127.1.244.193:43225/ using document root <none> and password file <none>
I20260807 04:23:08.859340 9841 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:08.859474 9841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:08.859790 9841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:08.862540 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "896594360f6e49c8af14a815c55bc4bd"
format_stamp: "Formatted at 2026-08-07 04:23:08 on dist-test-slave-zkpd"
server_key: "690de765cf8378a5e0e31b2e7262684d"
server_key_iv: "2682048021478b24fd594a85491063a1"
server_key_version: "encryptionkey@0"
I20260807 04:23:08.863328 9841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "896594360f6e49c8af14a815c55bc4bd"
format_stamp: "Formatted at 2026-08-07 04:23:08 on dist-test-slave-zkpd"
server_key: "690de765cf8378a5e0e31b2e7262684d"
server_key_iv: "2682048021478b24fd594a85491063a1"
server_key_version: "encryptionkey@0"
I20260807 04:23:08.868460 9841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260807 04:23:08.872112 9857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:08.873327 9841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:23:08.873494 9841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "896594360f6e49c8af14a815c55bc4bd"
format_stamp: "Formatted at 2026-08-07 04:23:08 on dist-test-slave-zkpd"
server_key: "690de765cf8378a5e0e31b2e7262684d"
server_key_iv: "2682048021478b24fd594a85491063a1"
server_key_version: "encryptionkey@0"
I20260807 04:23:08.873731 9841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:08.898361 9841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:08.902151 9841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:08.902381 9841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:08.904995 9841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:08.905103 9841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:08.905241 9841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:08.905308 9841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:23:08 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076588, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:08.927790 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:08.905294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41145 (local address 127.1.244.254:46539)
0807 04:23:08.905739 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:08.905747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:08.906817 (+ 1070us) server_negotiation.cc:407] Connection header received
0807 04:23:08.908987 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:08.908993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:08.909091 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:08.909221 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:08.911478 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.912638 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.915138 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.915920 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.916091 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:08.920507 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:08.920531 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:08.920537 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:08.920584 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:08.924019 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:08.925045 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:08.925052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:08.925058 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:08.925211 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:08.925830 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:08.925836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:08.925841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:08.926212 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:08.926370 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:08.927353 (+ 983us) server_negotiation.cc:299] Negotiation successful
0807 04:23:08.927522 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":223,"threads_started":1}
I20260807 04:23:08.929730 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:08.905946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.1.244.193:41145)
0807 04:23:08.906663 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:08.906707 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:08.908641 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:08.909442 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:08.909459 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:08.910312 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:08.911279 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:08.911303 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.912816 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.912820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:08.914955 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:08.914967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.916197 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.916205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:08.916442 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:08.917093 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:08.917114 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:08.920257 (+ 3143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:08.924198 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:08.924209 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:08.924234 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:08.924831 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:08.925340 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:08.925345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:08.925349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:08.925666 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:08.926386 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:08.926396 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:08.926919 (+ 523us) client_negotiation.cc:769] Sending connection context
0807 04:23:08.927278 (+ 359us) client_negotiation.cc:240] Negotiation successful
0807 04:23:08.927964 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":305,"threads_started":1}
I20260807 04:23:08.951853 9841 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:39049
I20260807 04:23:08.951917 9979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:39049 every 8 connection(s)
I20260807 04:23:08.952643 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:08.937749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41441 (local address 127.1.244.254:46539)
0807 04:23:08.937925 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:08.937932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:08.938509 (+ 577us) server_negotiation.cc:407] Connection header received
0807 04:23:08.938721 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:08.938726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:08.938802 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:08.938900 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:08.940602 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.941666 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.943085 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.943701 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.943860 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:08.946378 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:08.946384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:08.946389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:08.946456 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:08.949771 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:08.950439 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:08.950445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:08.950448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:08.950585 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:08.951057 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:08.951064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:08.951069 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:08.951367 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:08.951526 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:08.952075 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 04:23:08.952305 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 04:23:08.952737 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:08.937734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.1.244.193:41441)
0807 04:23:08.938393 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:08.938417 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:08.938568 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:08.939106 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:08.939114 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:08.939831 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:08.940402 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:08.940418 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.941792 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.941802 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:08.942938 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:08.942951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:08.944156 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:08.944161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:08.944297 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:08.945187 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:08.945206 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:08.946185 (+ 979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:08.949901 (+ 3716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:08.949914 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:08.949918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:08.950313 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:08.950703 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:08.950708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:08.950715 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:08.950875 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:08.951499 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:08.951504 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:08.951696 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 04:23:08.951862 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 04:23:08.952239 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":421,"threads_started":1}
I20260807 04:23:08.953264 9841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:08.955422 9980 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46539
I20260807 04:23:08.956229 9980 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:08.957060 9980 heartbeater.cc:511] Master 127.1.244.254:46539 requested a full tablet report, sending...
I20260807 04:23:08.959000 9766 ts_manager.cc:194] Registered new tserver with Master: 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049)
I20260807 04:23:08.960455 9766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:41145
I20260807 04:23:08.962594 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 9841
I20260807 04:23:08.962807 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:23:08.963485 2003 external_mini_cluster.cc:1428] Setting key 4327cd4fe5a9528fcac9310458484267
WARNING: no policy specified for oryx/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
I20260807 04:23:09.013309 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46539
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:09.342334 9984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:09.342737 9984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:09.342933 9984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:09.350873 9984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:09.350965 9984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:09.351069 9984 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:23:09.351214 9984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:09.360467 9984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:46539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:09.361583 9984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:09.362871 9984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:09.375937 9993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:09.375880 9991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:09.375841 9990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:09.376485 9984 server_base.cc:1034] running on GCE node
I20260807 04:23:09.377072 9984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:09.378023 9984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:09.379244 9984 hybrid_clock.cc:648] HybridClock initialized: now 1786076589379159 us; error 61 us; skew 500 ppm
I20260807 04:23:09.379535 9984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:09 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076589, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:09.382934 9984 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260807 04:23:09.384317 9984 webserver.cc:460] Webserver started at http://127.1.244.194:38613/ using document root <none> and password file <none>
I20260807 04:23:09.384990 9984 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:09.385102 9984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:09.385450 9984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:09.388378 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "7110402f92f04b17afb8d64ddb5baf09"
format_stamp: "Formatted at 2026-08-07 04:23:09 on dist-test-slave-zkpd"
server_key: "67d499167575d9995f2bcf016673de27"
server_key_iv: "b2e1b9e72847eb769cc931ff20f09b89"
server_key_version: "encryptionkey@0"
I20260807 04:23:09.389168 9984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "7110402f92f04b17afb8d64ddb5baf09"
format_stamp: "Formatted at 2026-08-07 04:23:09 on dist-test-slave-zkpd"
server_key: "67d499167575d9995f2bcf016673de27"
server_key_iv: "b2e1b9e72847eb769cc931ff20f09b89"
server_key_version: "encryptionkey@0"
I20260807 04:23:09.393970 9984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:23:09.397212 10000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:09.398408 9984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 04:23:09.398573 9984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "7110402f92f04b17afb8d64ddb5baf09"
format_stamp: "Formatted at 2026-08-07 04:23:09 on dist-test-slave-zkpd"
server_key: "67d499167575d9995f2bcf016673de27"
server_key_iv: "b2e1b9e72847eb769cc931ff20f09b89"
server_key_version: "encryptionkey@0"
I20260807 04:23:09.398779 9984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:09.419876 9984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:09.424041 9984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:09.424304 9984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:09.426690 9984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:09.426802 9984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:09.427004 9984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:09.427085 9984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:23:09 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076589, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:09.451738 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.427510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:57355 (local address 127.1.244.254:46539)
0807 04:23:09.427709 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:09.427715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:09.429202 (+ 1487us) server_negotiation.cc:407] Connection header received
0807 04:23:09.431172 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:09.431178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:09.431277 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:09.431417 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:09.433811 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.434853 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.437555 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.438133 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.438268 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.443043 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:09.443050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:09.443054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:09.443097 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:09.447364 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.448546 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.448555 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.448561 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.448727 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.449343 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.449350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.449355 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.449806 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:09.449985 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:09.450933 (+ 948us) server_negotiation.cc:299] Negotiation successful
0807 04:23:09.451205 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:23:09.452533 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.428117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.1.244.194:57355)
0807 04:23:09.429009 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:09.429066 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:09.430879 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:09.431721 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:09.431737 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:09.432612 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:09.433580 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.433612 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.435031 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.435040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.437386 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.437399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.438501 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.438509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.438710 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.439464 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:09.439511 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:09.442802 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:09.447528 (+ 4726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.447539 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.447557 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.448347 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.448864 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.448871 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.448876 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.449194 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.449973 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:09.449983 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:09.450438 (+ 455us) client_negotiation.cc:769] Sending connection context
0807 04:23:09.450780 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 04:23:09.451331 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":391,"threads_started":1}
I20260807 04:23:09.476548 9984 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:43549
I20260807 04:23:09.476553 10122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:43549 every 8 connection(s)
I20260807 04:23:09.478080 9984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:09.480037 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.460538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.1.244.194:49077)
0807 04:23:09.461567 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:09.461628 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:09.461814 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:09.464171 (+ 2357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:09.464176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:09.464838 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:09.465297 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.465315 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.467174 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.467183 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.469445 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.469466 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.470775 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.470780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.470943 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.471662 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:09.471679 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:09.472839 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:09.476931 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.476935 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.476937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.477360 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.477795 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.477798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.477801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.477962 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.478764 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:09.478771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:09.479046 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 04:23:09.479192 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 04:23:09.479538 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":469,"threads_started":1}
I20260807 04:23:09.480492 10123 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46539
I20260807 04:23:09.480571 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.462943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49077 (local address 127.1.244.254:46539)
0807 04:23:09.463812 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:09.463817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:09.463836 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:23:09.463925 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:09.463930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:09.464006 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:09.464115 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:09.465538 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.466955 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.469657 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.470517 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.470700 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.473072 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:09.473077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:09.473080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:09.473121 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:09.476777 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.477507 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.477513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.477517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.477684 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.478093 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.478101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.478106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.478579 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:09.480060 (+ 1481us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:09.480188 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 04:23:09.480357 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:23:09.481069 10123 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:09.481890 10123 heartbeater.cc:511] Master 127.1.244.254:46539 requested a full tablet report, sending...
I20260807 04:23:09.483410 9766 ts_manager.cc:194] Registered new tserver with Master: 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549)
I20260807 04:23:09.484609 9766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:57355
I20260807 04:23:09.485672 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 9984
I20260807 04:23:09.485860 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:23:09.486416 2003 external_mini_cluster.cc:1428] Setting key 4dfeb33c5f5ff3b37501e52b4c59f40d
WARNING: no policy specified for oryx/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
I20260807 04:23:09.536675 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46539
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:09.858727 10127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:09.858980 10127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:09.859153 10127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:09.866161 10127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:09.866271 10127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:09.866428 10127 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 04:23:09.866533 10127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:09.875998 10127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.244.254:46539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:09.877189 10127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:09.878482 10127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:09.891489 10133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:09.891533 10134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:09.891556 10136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:09.892378 10127 server_base.cc:1034] running on GCE node
I20260807 04:23:09.893014 10127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:09.893872 10127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:09.895124 10127 hybrid_clock.cc:648] HybridClock initialized: now 1786076589895060 us; error 55 us; skew 500 ppm
I20260807 04:23:09.895416 10127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:09 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076589, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:09.898486 10127 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
I20260807 04:23:09.899984 10127 webserver.cc:460] Webserver started at http://127.1.244.195:44393/ using document root <none> and password file <none>
I20260807 04:23:09.900626 10127 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:09.900780 10127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:09.901118 10127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:09.903818 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "b0a5bf6e41144c80816cbab240e38b71"
format_stamp: "Formatted at 2026-08-07 04:23:09 on dist-test-slave-zkpd"
server_key: "8b70ec17c73786213547aa65f07cb2df"
server_key_iv: "df39b8c83e28dbc621e2db681d55b87c"
server_key_version: "encryptionkey@0"
I20260807 04:23:09.904697 10127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "b0a5bf6e41144c80816cbab240e38b71"
format_stamp: "Formatted at 2026-08-07 04:23:09 on dist-test-slave-zkpd"
server_key: "8b70ec17c73786213547aa65f07cb2df"
server_key_iv: "df39b8c83e28dbc621e2db681d55b87c"
server_key_version: "encryptionkey@0"
I20260807 04:23:09.909413 10127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 04:23:09.912705 10143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:09.913919 10127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:09.914089 10127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "b0a5bf6e41144c80816cbab240e38b71"
format_stamp: "Formatted at 2026-08-07 04:23:09 on dist-test-slave-zkpd"
server_key: "8b70ec17c73786213547aa65f07cb2df"
server_key_iv: "df39b8c83e28dbc621e2db681d55b87c"
server_key_version: "encryptionkey@0"
I20260807 04:23:09.914301 10127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:09.940327 10127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:09.944075 10127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:09.944355 10127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:09.946792 10127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:09.946936 10127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:09.947091 10127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:09.947170 10127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:23:09 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076589, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:09.963238 9980 heartbeater.cc:503] Master 127.1.244.254:46539 was elected leader, sending a full tablet report...
I20260807 04:23:09.973024 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.947588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52763 (local address 127.1.244.254:46539)
0807 04:23:09.947785 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:09.947791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:09.949270 (+ 1479us) server_negotiation.cc:407] Connection header received
0807 04:23:09.951334 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:09.951342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:09.951435 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:09.951506 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:09.954266 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.955445 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.958059 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.958643 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.958783 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.964600 (+ 5817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:09.964606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:09.964610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:09.964646 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:09.968523 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.969464 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.969470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.969476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.969661 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.970367 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.970375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.970380 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.970852 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:09.971019 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:09.972290 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0807 04:23:09.972573 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 04:23:09.974733 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.948185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.1.244.195:52763)
0807 04:23:09.949078 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:09.949132 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:09.950984 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:09.952015 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:09.952028 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:09.952803 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:09.954036 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.954065 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.955640 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.955647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.957882 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.957919 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.959001 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.959046 (+ 45us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.959653 (+ 607us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.960482 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:09.960522 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:09.964331 (+ 3809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:09.968662 (+ 4331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.968673 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.968696 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.969309 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.969824 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.969831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.969841 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.970192 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.971111 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:09.971123 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:09.971717 (+ 594us) client_negotiation.cc:769] Sending connection context
0807 04:23:09.972125 (+ 408us) client_negotiation.cc:240] Negotiation successful
0807 04:23:09.972797 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":395,"threads_started":1}
I20260807 04:23:09.990715 10127 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41151
I20260807 04:23:09.990763 10265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41151 every 8 connection(s)
I20260807 04:23:09.992111 10127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:09.994234 10266 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46539
I20260807 04:23:09.994483 10266 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:09.995343 10266 heartbeater.cc:511] Master 127.1.244.254:46539 requested a full tablet report, sending...
I20260807 04:23:09.996860 9766 ts_manager.cc:194] Registered new tserver with Master: b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151)
I20260807 04:23:09.997022 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 10127
I20260807 04:23:09.997175 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:23:09.997180 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.982205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45679 (local address 127.1.244.254:46539)
0807 04:23:09.982439 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:09.982448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:09.983015 (+ 567us) server_negotiation.cc:407] Connection header received
0807 04:23:09.983271 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:09.983277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:09.983367 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:09.983472 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:09.984665 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.985765 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.987941 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.988548 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.988687 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.990991 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:09.990996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:09.990999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:09.991027 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:09.994273 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.995144 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.995148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.995151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.995272 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:09.995647 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:09.995650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:09.995652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:09.995940 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:09.996057 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:09.996547 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 04:23:09.996756 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:23:09.997241 10247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:09.982225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.1.244.195:45679)
0807 04:23:09.982902 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:09.982928 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:09.983075 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:09.983505 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:09.983510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:09.983999 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:09.984527 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.984542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.985958 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.985963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.987736 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:09.987752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:09.988729 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:09.988736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:09.988914 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:09.989742 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:09.989761 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:09.990745 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:09.994448 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.994453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.994457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.995029 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.995387 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:09.995392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:09.995396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:09.995550 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:09.996058 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:09.996064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:09.996250 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 04:23:09.996427 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 04:23:09.996771 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":362,"threads_started":1}
I20260807 04:23:09.997779 2003 external_mini_cluster.cc:1428] Setting key a15ac63ded1dac0b1f6d804fda5698f5
I20260807 04:23:09.998039 9766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:52763
I20260807 04:23:10.001989 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 04:23:10 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 04:23:10 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:10 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:10.487581 10123 heartbeater.cc:503] Master 127.1.244.254:46539 was elected leader, sending a full tablet report...
I20260807 04:23:11.000982 10266 heartbeater.cc:503] Master 127.1.244.254:46539 was elected leader, sending a full tablet report...
W20260807 04:23:11.184201 10275 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.132s user 0.006s sys 0.012s
W20260807 04:23:11.185401 9865 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:10.046253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.1.244.254:46539)
0807 04:23:10.046532 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:10.046539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:10.046858 (+ 319us) server_negotiation.cc:407] Connection header received
0807 04:23:10.047056 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:10.047060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:10.047123 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:10.047206 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:10.048563 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:10.049285 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:10.050440 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:10.050853 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:10.050941 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.185147 (+1134206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36206: BlockingRecv error: recv got EOF from 127.0.0.1:36206 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 07 04:23:11 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:11.203666 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.189643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.1.244.254:46539)
0807 04:23:11.189878 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.189884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.190162 (+ 278us) server_negotiation.cc:407] Connection header received
0807 04:23:11.190376 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.190381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.190473 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.190586 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:11.191986 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.192737 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.194003 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.194417 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.194552 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.198338 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:11.198349 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:11.198352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:11.198384 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:11.200978 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:11.201687 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:11.201694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:11.201697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:11.201826 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:11.202246 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:11.202249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:11.202251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:11.202584 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:11.202807 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.203159 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.203335 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:23:11.207547 9765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:23:11.214442 9765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:11.239909 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.227094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.1.244.195:41151)
0807 04:23:11.228221 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.228234 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.229312 (+ 1078us) server_negotiation.cc:407] Connection header received
0807 04:23:11.229412 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.229419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.229631 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.229838 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.231066 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.232770 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.236442 (+ 3672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.238437 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.238615 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.238894 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.239397 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.239581 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":605,"threads_started":1}
I20260807 04:23:11.240667 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.226531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41151 (local address 127.0.0.1:35386)
0807 04:23:11.229047 (+ 2516us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.229067 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.229190 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.230112 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.230119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.230153 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.230692 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.230706 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.233011 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.233019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.236193 (+ 3174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.236215 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.238671 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.238680 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.238906 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.238971 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.239103 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.239303 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2341,"mutex_wait_us":44,"thread_start_us":2190,"threads_started":1}
I20260807 04:23:11.241722 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.226158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:43549 (local address 127.0.0.1:40418)
0807 04:23:11.226888 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.226939 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.227116 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.228395 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.228406 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.228449 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.229052 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.229073 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.231490 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.231499 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.239002 (+ 7503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.239013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.240930 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.240938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.241123 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.241159 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.241231 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.241357 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":4087,"thread_start_us":309,"threads_started":1}
I20260807 04:23:11.242709 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.226449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.1.244.194:43549)
0807 04:23:11.227729 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.227738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.227766 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:23:11.227867 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.227877 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.228163 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.228364 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.229320 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.231294 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.239175 (+ 7881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.240657 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.242035 (+ 1378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.242225 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.242307 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.242482 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":930,"threads_started":1}
I20260807 04:23:11.244621 10194 tablet_service.cc:1467] Processing CreateTablet for tablet 926e8d2918a34f14b7ca05cc11eb9398 (DEFAULT_TABLE table=test-table [id=2613f62572654029b2172cfce0916d8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:11.246192 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 926e8d2918a34f14b7ca05cc11eb9398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.245913 10051 tablet_service.cc:1467] Processing CreateTablet for tablet 926e8d2918a34f14b7ca05cc11eb9398 (DEFAULT_TABLE table=test-table [id=2613f62572654029b2172cfce0916d8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:11.247289 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.227929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:39049 (local address 127.0.0.1:46342)
0807 04:23:11.238491 (+ 10562us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.238505 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.238634 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.240202 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.240209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.240234 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.240611 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.240626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.242487 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.242494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.244444 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.244460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.246561 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.246568 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.246737 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.246771 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.246853 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.246990 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10475,"thread_start_us":10798,"threads_started":1}
I20260807 04:23:11.247326 10051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 926e8d2918a34f14b7ca05cc11eb9398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.248028 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.228224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.1.244.193:39049)
0807 04:23:11.239398 (+ 11174us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.239407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.239435 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:23:11.239526 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.239533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.239744 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.239984 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.240812 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.242335 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.244637 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.246078 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.246224 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.246428 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.247540 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.247704 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11017,"thread_start_us":10867,"threads_started":1}
I20260807 04:23:11.251379 9908 tablet_service.cc:1467] Processing CreateTablet for tablet 926e8d2918a34f14b7ca05cc11eb9398 (DEFAULT_TABLE table=test-table [id=2613f62572654029b2172cfce0916d8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:11.252696 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 926e8d2918a34f14b7ca05cc11eb9398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.259510 10293 tablet_bootstrap.cc:492] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Bootstrap starting.
I20260807 04:23:11.261438 10294 tablet_bootstrap.cc:492] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Bootstrap starting.
I20260807 04:23:11.263353 10293 tablet_bootstrap.cc:654] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.264672 10293 log.cc:824] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:11.264899 10294 tablet_bootstrap.cc:654] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.266192 10294 log.cc:824] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:11.267364 10296 tablet_bootstrap.cc:492] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Bootstrap starting.
I20260807 04:23:11.267977 10293 tablet_bootstrap.cc:492] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: No bootstrap required, opened a new log
I20260807 04:23:11.268254 10293 ts_tablet_manager.cc:1397] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 04:23:11.269014 10294 tablet_bootstrap.cc:492] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: No bootstrap required, opened a new log
I20260807 04:23:11.269201 10294 ts_tablet_manager.cc:1397] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:23:11.270947 10296 tablet_bootstrap.cc:654] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.272135 10296 log.cc:824] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:11.275218 10296 tablet_bootstrap.cc:492] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: No bootstrap required, opened a new log
I20260807 04:23:11.275468 10296 ts_tablet_manager.cc:1397] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 04:23:11.277087 10294 raft_consensus.cc:348] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.277531 10294 raft_consensus.cc:374] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.277695 10294 raft_consensus.cc:729] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.278436 10294 consensus_queue.cc:260] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.278870 10293 raft_consensus.cc:348] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.279289 10293 raft_consensus.cc:374] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.279393 10293 raft_consensus.cc:729] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7110402f92f04b17afb8d64ddb5baf09, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.280005 10293 consensus_queue.cc:260] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.281677 10293 ts_tablet_manager.cc:1428] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 04:23:11.283195 10294 ts_tablet_manager.cc:1428] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260807 04:23:11.285990 10296 raft_consensus.cc:348] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.286399 10296 raft_consensus.cc:374] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.286516 10296 raft_consensus.cc:729] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896594360f6e49c8af14a815c55bc4bd, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.287119 10296 consensus_queue.cc:260] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.288887 10296 ts_tablet_manager.cc:1428] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
W20260807 04:23:11.456579 9981 tablet.cc:2369] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:23:11.481348 10124 tablet.cc:2369] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:23:11.494704 10267 tablet.cc:2369] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:23:11.548415 10299 raft_consensus.cc:482] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:11.548794 10299 raft_consensus.cc:504] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.550483 10299 leader_election.cc:288] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549), 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049)
I20260807 04:23:11.557054 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.551016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:40119 (local address 127.1.244.194:43549)
0807 04:23:11.551267 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.551274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.551511 (+ 237us) server_negotiation.cc:407] Connection header received
0807 04:23:11.551786 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.551795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.551954 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.552056 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.552681 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.553406 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.555211 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.556127 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.556315 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.556393 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.556666 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.556793 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:23:11.557157 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.550788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:43549 (local address 127.1.244.195:40119)
0807 04:23:11.551371 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.551387 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.551588 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.552138 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.552144 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.552170 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.552533 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.552542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.553538 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.553547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.555054 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.555066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.556338 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.556353 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.556506 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.556530 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.556637 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.556764 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":295,"threads_started":1}
I20260807 04:23:11.557632 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.551835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47757 (local address 127.1.244.193:39049)
0807 04:23:11.552137 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.552144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.552332 (+ 188us) server_negotiation.cc:407] Connection header received
0807 04:23:11.552550 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.552554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.552619 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.552779 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.553307 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.554043 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.555837 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.556682 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.556877 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.556951 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.557323 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.557453 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 04:23:11.558205 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.551538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:39049 (local address 127.1.244.195:47757)
0807 04:23:11.552230 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.552247 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.552402 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.552795 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.552800 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.552823 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.553139 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.553147 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.554188 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.554201 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.555682 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.555695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.556919 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.556925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.557078 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.557100 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.557828 (+ 728us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.557962 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":373,"threads_started":1}
I20260807 04:23:11.558156 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "926e8d2918a34f14b7ca05cc11eb9398" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09" is_pre_election: true
I20260807 04:23:11.558712 10072 raft_consensus.cc:2454] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 0.
I20260807 04:23:11.559295 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "926e8d2918a34f14b7ca05cc11eb9398" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd" is_pre_election: true
I20260807 04:23:11.559455 10144 leader_election.cc:302] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:11.559764 9931 raft_consensus.cc:2454] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 0.
I20260807 04:23:11.559965 10299 raft_consensus.cc:2781] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:11.560142 10299 raft_consensus.cc:482] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:11.560266 10299 raft_consensus.cc:3037] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.562516 10299 raft_consensus.cc:504] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.563107 10299 leader_election.cc:288] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 election: Requested vote from peers 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549), 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049)
I20260807 04:23:11.563591 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "926e8d2918a34f14b7ca05cc11eb9398" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09"
I20260807 04:23:11.563643 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "926e8d2918a34f14b7ca05cc11eb9398" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd"
I20260807 04:23:11.563817 10072 raft_consensus.cc:3037] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.563827 9931 raft_consensus.cc:3037] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.567135 10072 raft_consensus.cc:2454] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 1.
I20260807 04:23:11.567134 9931 raft_consensus.cc:2454] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 1.
I20260807 04:23:11.567555 10144 leader_election.cc:302] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:11.567844 10299 raft_consensus.cc:2781] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:11.568315 10299 raft_consensus.cc:686] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 1 LEADER]: Becoming Leader. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Running, Role: LEADER
I20260807 04:23:11.568779 10299 consensus_queue.cc:237] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } }
I20260807 04:23:11.572685 9765 catalog_manager.cc:5495] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "b0a5bf6e41144c80816cbab240e38b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } health_report { overall_health: UNKNOWN } } }
I20260807 04:23:11.604476 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.599256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.1.244.195:41151)
0807 04:23:11.599477 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.599483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.599608 (+ 125us) server_negotiation.cc:407] Connection header received
0807 04:23:11.599796 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.599800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.599876 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.600059 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:11.600362 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.601049 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.602323 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.602757 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.602871 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.603331 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:11.603829 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:11.604018 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.604115 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.604198 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:23:11.612118 10072 raft_consensus.cc:1261] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Refusing update from remote peer b0a5bf6e41144c80816cbab240e38b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:11.612121 9931 raft_consensus.cc:1261] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Refusing update from remote peer b0a5bf6e41144c80816cbab240e38b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:11.613018 10304 consensus_queue.cc:1035] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:11.613379 10299 consensus_queue.cc:1035] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:11.628334 10309 mvcc.cc:205] Tried to move back new op lower bound from 7315769719234052096 to 7315769719069573120. Current Snapshot: MvccSnapshot[applied={T|T < 7315769719234052096}]
I20260807 04:23:11.639770 9765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:11.640064 9765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:11.643730 9765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Sending DeleteTablet for 3 replicas of tablet 926e8d2918a34f14b7ca05cc11eb9398
I20260807 04:23:11.644809 10194 tablet_service.cc:1514] Processing DeleteTablet for tablet 926e8d2918a34f14b7ca05cc11eb9398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:11 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260807 04:23:11.644902 10051 tablet_service.cc:1514] Processing DeleteTablet for tablet 926e8d2918a34f14b7ca05cc11eb9398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:11 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40418
I20260807 04:23:11.644991 9908 tablet_service.cc:1514] Processing DeleteTablet for tablet 926e8d2918a34f14b7ca05cc11eb9398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:11 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46342
I20260807 04:23:11.645838 10317 tablet_replica.cc:330] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: stopping tablet replica
I20260807 04:23:11.646003 10319 tablet_replica.cc:330] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: stopping tablet replica
I20260807 04:23:11.646445 10319 raft_consensus.cc:2229] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:11.646432 10317 raft_consensus.cc:2229] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:23:11.646961 10319 raft_consensus.cc:2258] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:11.646970 9765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 04:23:11.647291 10317 raft_consensus.cc:2258] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 04:23:11.648047 9765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:11.649243 10319 ts_tablet_manager.cc:1905] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:11.649326 10317 ts_tablet_manager.cc:1905] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:11.655853 10050 tablet_service.cc:1467] Processing CreateTablet for tablet c0b0178b7b1848fc8d7ed418e88ed855 (DEFAULT_TABLE table=test-table [id=325803a030cc4b0cb0050388ee960d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:11.656188 10319 ts_tablet_manager.cc:1918] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:11.656366 10319 log.cc:1196] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/926e8d2918a34f14b7ca05cc11eb9398
I20260807 04:23:11.656538 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b0178b7b1848fc8d7ed418e88ed855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.656889 10319 ts_tablet_manager.cc:1939] T 926e8d2918a34f14b7ca05cc11eb9398 P 896594360f6e49c8af14a815c55bc4bd: Deleting consensus metadata
I20260807 04:23:11.658015 10317 ts_tablet_manager.cc:1918] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:11.658196 10317 log.cc:1196] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/926e8d2918a34f14b7ca05cc11eb9398
I20260807 04:23:11.658797 10317 ts_tablet_manager.cc:1939] T 926e8d2918a34f14b7ca05cc11eb9398 P b0a5bf6e41144c80816cbab240e38b71: Deleting consensus metadata
I20260807 04:23:11.658834 9750 catalog_manager.cc:4841] TS 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049): tablet 926e8d2918a34f14b7ca05cc11eb9398 (table test-table [id=2613f62572654029b2172cfce0916d8f]) successfully deleted
I20260807 04:23:11.659132 9908 tablet_service.cc:1467] Processing CreateTablet for tablet c0b0178b7b1848fc8d7ed418e88ed855 (DEFAULT_TABLE table=test-table [id=325803a030cc4b0cb0050388ee960d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:11.659554 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b0178b7b1848fc8d7ed418e88ed855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.660876 10318 tablet_replica.cc:330] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: stopping tablet replica
I20260807 04:23:11.661264 10318 raft_consensus.cc:2229] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:11.661679 10318 raft_consensus.cc:2258] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:11.664618 10293 tablet_bootstrap.cc:492] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Bootstrap starting.
I20260807 04:23:11.665536 10296 tablet_bootstrap.cc:492] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Bootstrap starting.
I20260807 04:23:11.665812 9751 catalog_manager.cc:4841] TS b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151): tablet 926e8d2918a34f14b7ca05cc11eb9398 (table test-table [id=2613f62572654029b2172cfce0916d8f]) successfully deleted
I20260807 04:23:11.666234 10293 tablet_bootstrap.cc:654] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.667119 10296 tablet_bootstrap.cc:654] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.667549 10194 tablet_service.cc:1467] Processing CreateTablet for tablet c0b0178b7b1848fc8d7ed418e88ed855 (DEFAULT_TABLE table=test-table [id=325803a030cc4b0cb0050388ee960d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:11.668033 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b0178b7b1848fc8d7ed418e88ed855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.669783 10296 tablet_bootstrap.cc:492] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: No bootstrap required, opened a new log
I20260807 04:23:11.669917 10296 ts_tablet_manager.cc:1397] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260807 04:23:11.670769 10296 raft_consensus.cc:348] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.671028 10296 raft_consensus.cc:374] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.671146 10296 raft_consensus.cc:729] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896594360f6e49c8af14a815c55bc4bd, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.671502 10296 consensus_queue.cc:260] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.672609 10293 tablet_bootstrap.cc:492] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: No bootstrap required, opened a new log
I20260807 04:23:11.672771 10293 ts_tablet_manager.cc:1397] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260807 04:23:11.672879 10296 ts_tablet_manager.cc:1428] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 04:23:11.673422 10293 raft_consensus.cc:348] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.673611 10293 raft_consensus.cc:374] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.673779 10293 raft_consensus.cc:729] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7110402f92f04b17afb8d64ddb5baf09, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.674078 10293 consensus_queue.cc:260] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.674625 10293 ts_tablet_manager.cc:1428] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 04:23:11.675046 10294 tablet_bootstrap.cc:492] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Bootstrap starting.
I20260807 04:23:11.676059 10318 ts_tablet_manager.cc:1905] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:11.676810 10294 tablet_bootstrap.cc:654] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.684881 10318 ts_tablet_manager.cc:1918] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:11.685055 10318 log.cc:1196] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/926e8d2918a34f14b7ca05cc11eb9398
I20260807 04:23:11.685628 10318 ts_tablet_manager.cc:1939] T 926e8d2918a34f14b7ca05cc11eb9398 P 7110402f92f04b17afb8d64ddb5baf09: Deleting consensus metadata
I20260807 04:23:11.686059 10294 tablet_bootstrap.cc:492] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: No bootstrap required, opened a new log
I20260807 04:23:11.686225 10294 ts_tablet_manager.cc:1397] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260807 04:23:11.687422 9750 catalog_manager.cc:4841] TS 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549): tablet 926e8d2918a34f14b7ca05cc11eb9398 (table test-table [id=2613f62572654029b2172cfce0916d8f]) successfully deleted
I20260807 04:23:11.687330 10294 raft_consensus.cc:348] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.687613 10294 raft_consensus.cc:374] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.687772 10294 raft_consensus.cc:729] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.688078 10294 consensus_queue.cc:260] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.688805 10294 ts_tablet_manager.cc:1428] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:23:11.714396 10304 raft_consensus.cc:482] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:11.714640 10304 raft_consensus.cc:504] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.715276 10304 leader_election.cc:288] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049), 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549)
I20260807 04:23:11.716008 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b0178b7b1848fc8d7ed418e88ed855" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd" is_pre_election: true
I20260807 04:23:11.715997 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b0178b7b1848fc8d7ed418e88ed855" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09" is_pre_election: true
I20260807 04:23:11.716256 10072 raft_consensus.cc:2454] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 0.
I20260807 04:23:11.716262 9931 raft_consensus.cc:2454] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 0.
I20260807 04:23:11.716701 10144 leader_election.cc:302] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 896594360f6e49c8af14a815c55bc4bd, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:11.717037 10304 raft_consensus.cc:2781] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:11.717159 10304 raft_consensus.cc:482] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:11.717268 10304 raft_consensus.cc:3037] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.720543 10304 raft_consensus.cc:504] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.721204 10304 leader_election.cc:288] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 election: Requested vote from peers 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049), 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549)
I20260807 04:23:11.721652 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b0178b7b1848fc8d7ed418e88ed855" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd"
I20260807 04:23:11.721849 9931 raft_consensus.cc:3037] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.721807 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b0178b7b1848fc8d7ed418e88ed855" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09"
I20260807 04:23:11.722002 10072 raft_consensus.cc:3037] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.724112 9931 raft_consensus.cc:2454] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 1.
I20260807 04:23:11.724606 10144 leader_election.cc:302] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 896594360f6e49c8af14a815c55bc4bd, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:11.725001 10072 raft_consensus.cc:2454] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 1.
I20260807 04:23:11.725322 10304 raft_consensus.cc:2781] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:11.725518 10304 raft_consensus.cc:686] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 1 LEADER]: Becoming Leader. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Running, Role: LEADER
I20260807 04:23:11.726042 10304 consensus_queue.cc:237] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.729621 9765 catalog_manager.cc:5495] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "b0a5bf6e41144c80816cbab240e38b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } health_report { overall_health: HEALTHY } } }
Aug 07 04:23:11 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076588, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.254@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:11.778487 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.762357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.1.244.254:46539)
0807 04:23:11.763189 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.763195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.763224 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:23:11.763279 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.763285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.763368 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.763491 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:11.764783 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.765985 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.767815 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.768309 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.768414 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.772581 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:11.772585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:11.772588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:11.772617 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:11.775441 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:11.776313 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:11.776316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:11.776319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:11.776435 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:11.776871 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:11.776877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:11.776880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:11.777231 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:11.777348 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.777861 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.778225 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":691,"thread_start_us":526,"threads_started":1}
I20260807 04:23:11.778568 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.762216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46539 (local address 127.0.0.1:59382)
0807 04:23:11.762962 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.762989 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.763142 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.763565 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.763568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.764377 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:11.764607 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.764617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.766159 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.766164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.767676 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.767691 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.768481 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.768488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.768688 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.769402 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:11.769426 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:11.772375 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:11.775572 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:11.775585 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:11.775593 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:11.776208 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:11.776541 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:11.776549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:11.776553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:11.776712 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:11.777362 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:11.777371 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:11.777575 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.777760 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.778090 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":348,"threads_started":1}
I20260807 04:23:11.783792 9765 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:59382:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 04:23:11.785089 9765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:11.792476 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 0627bb59c3884d1c8e8f36a42c87104e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=02eb172c9aeb445e88284404c367824d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:23:11.792477 9908 tablet_service.cc:1467] Processing CreateTablet for tablet 0627bb59c3884d1c8e8f36a42c87104e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=02eb172c9aeb445e88284404c367824d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:23:11.792555 10194 tablet_service.cc:1467] Processing CreateTablet for tablet 0627bb59c3884d1c8e8f36a42c87104e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=02eb172c9aeb445e88284404c367824d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 04:23:11.793061 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0627bb59c3884d1c8e8f36a42c87104e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.793092 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0627bb59c3884d1c8e8f36a42c87104e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.793126 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0627bb59c3884d1c8e8f36a42c87104e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:11.800383 10294 tablet_bootstrap.cc:492] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: Bootstrap starting.
I20260807 04:23:11.800397 10293 tablet_bootstrap.cc:492] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: Bootstrap starting.
I20260807 04:23:11.802212 10293 tablet_bootstrap.cc:654] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.802222 10294 tablet_bootstrap.cc:654] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.804174 10296 tablet_bootstrap.cc:492] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: Bootstrap starting.
I20260807 04:23:11.805900 10293 tablet_bootstrap.cc:492] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: No bootstrap required, opened a new log
I20260807 04:23:11.806066 10293 ts_tablet_manager.cc:1397] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260807 04:23:11.806279 10296 tablet_bootstrap.cc:654] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:11.806818 10293 raft_consensus.cc:348] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.808095 10293 raft_consensus.cc:374] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.808240 10293 raft_consensus.cc:729] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7110402f92f04b17afb8d64ddb5baf09, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.808562 10293 consensus_queue.cc:260] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.809276 10293 ts_tablet_manager.cc:1428] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 04:23:11.809101 10300 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.809507 10300 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:11.812846 10296 tablet_bootstrap.cc:492] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: No bootstrap required, opened a new log
I20260807 04:23:11.813057 10296 ts_tablet_manager.cc:1397] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260807 04:23:11.813606 10294 tablet_bootstrap.cc:492] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: No bootstrap required, opened a new log
I20260807 04:23:11.813796 10294 ts_tablet_manager.cc:1397] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260807 04:23:11.813843 10296 raft_consensus.cc:348] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.814160 10296 raft_consensus.cc:374] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.814599 10296 raft_consensus.cc:729] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896594360f6e49c8af14a815c55bc4bd, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.814604 10294 raft_consensus.cc:348] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.814868 10294 raft_consensus.cc:374] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:11.815007 10294 raft_consensus.cc:729] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Initialized, Role: FOLLOWER
I20260807 04:23:11.815068 10296 consensus_queue.cc:260] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.815276 10294 consensus_queue.cc:260] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.815608 10301 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.816002 10301 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:11.815819 10304 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.816133 10304 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:11.816642 10294 ts_tablet_manager.cc:1428] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 04:23:11.817642 10296 ts_tablet_manager.cc:1428] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260807 04:23:11.880687 10301 raft_consensus.cc:482] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:11.880903 10301 raft_consensus.cc:504] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.882467 10301 leader_election.cc:288] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549), b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151)
I20260807 04:23:11.893077 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.882865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:35311 (local address 127.1.244.194:43549)
0807 04:23:11.883067 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.883073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.885633 (+ 2560us) server_negotiation.cc:407] Connection header received
0807 04:23:11.885956 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.885962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.886044 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.886115 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.887012 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.888079 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.891214 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.892109 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.892302 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.892380 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.892672 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.892803 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:23:11.893172 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.883193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:43549 (local address 127.1.244.193:35311)
0807 04:23:11.885538 (+ 2345us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.885556 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.885724 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.886312 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.886317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.886346 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.886784 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.886792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.888257 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.888263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.891020 (+ 2757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.891033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.892329 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.892336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.892508 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.892533 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.892642 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.892769 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2228,"mutex_wait_us":210,"thread_start_us":2042,"threads_started":1}
I20260807 04:23:11.894126 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.883607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:46365 (local address 127.1.244.195:41151)
0807 04:23:11.883905 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.883914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.885352 (+ 1438us) server_negotiation.cc:407] Connection header received
0807 04:23:11.885663 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.885669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.885797 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.885925 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:11.886664 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.888051 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.891174 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.892529 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.892721 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.892800 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.893665 (+ 865us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.893809 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260807 04:23:11.894156 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.883029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41151 (local address 127.1.244.193:46365)
0807 04:23:11.885236 (+ 2207us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.885259 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.885480 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.885945 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.885950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.885975 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:11.886473 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.886492 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.888220 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.888226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.891018 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.891033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.893378 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.893383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.893504 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.893527 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.893636 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.893765 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2050,"mutex_wait_us":161,"thread_start_us":1900,"threads_started":1}
I20260807 04:23:11.894054 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0627bb59c3884d1c8e8f36a42c87104e" candidate_uuid: "896594360f6e49c8af14a815c55bc4bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09" is_pre_election: true
I20260807 04:23:11.894337 10072 raft_consensus.cc:2454] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 896594360f6e49c8af14a815c55bc4bd in term 0.
I20260807 04:23:11.894930 9858 leader_election.cc:302] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, 896594360f6e49c8af14a815c55bc4bd; no voters:
I20260807 04:23:11.895306 10301 raft_consensus.cc:2781] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:11.895215 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0627bb59c3884d1c8e8f36a42c87104e" candidate_uuid: "896594360f6e49c8af14a815c55bc4bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a5bf6e41144c80816cbab240e38b71" is_pre_election: true
I20260807 04:23:11.895439 10301 raft_consensus.cc:482] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:11.895537 10301 raft_consensus.cc:3037] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.895658 10215 raft_consensus.cc:2454] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 896594360f6e49c8af14a815c55bc4bd in term 0.
I20260807 04:23:11.897975 10301 raft_consensus.cc:504] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.898610 10301 leader_election.cc:288] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [CANDIDATE]: Term 1 election: Requested vote from peers 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549), b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151)
I20260807 04:23:11.899030 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0627bb59c3884d1c8e8f36a42c87104e" candidate_uuid: "896594360f6e49c8af14a815c55bc4bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09"
I20260807 04:23:11.899160 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0627bb59c3884d1c8e8f36a42c87104e" candidate_uuid: "896594360f6e49c8af14a815c55bc4bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a5bf6e41144c80816cbab240e38b71"
I20260807 04:23:11.899273 10072 raft_consensus.cc:3037] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.899369 10215 raft_consensus.cc:3037] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:11.902245 10072 raft_consensus.cc:2454] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 896594360f6e49c8af14a815c55bc4bd in term 1.
I20260807 04:23:11.902631 9858 leader_election.cc:302] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, 896594360f6e49c8af14a815c55bc4bd; no voters:
I20260807 04:23:11.902789 10215 raft_consensus.cc:2454] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 896594360f6e49c8af14a815c55bc4bd in term 1.
I20260807 04:23:11.902942 10301 raft_consensus.cc:2781] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:11.903491 10301 raft_consensus.cc:686] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [term 1 LEADER]: Becoming Leader. State: Replica: 896594360f6e49c8af14a815c55bc4bd, State: Running, Role: LEADER
I20260807 04:23:11.903954 10301 consensus_queue.cc:237] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:11.906309 10336 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: TxnStatusTablet state changed. Reason: New leader 896594360f6e49c8af14a815c55bc4bd. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.906714 10336 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:23:11.907392 10338 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 04:23:11.907622 10338 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 04:23:11.907601 9766 catalog_manager.cc:5495] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 896594360f6e49c8af14a815c55bc4bd (127.1.244.193). New cstate: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } health_report { overall_health: UNKNOWN } } }
I20260807 04:23:11.945379 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.937239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.1.244.193:39049)
0807 04:23:11.937453 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:11.937460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:11.937833 (+ 373us) server_negotiation.cc:407] Connection header received
0807 04:23:11.938124 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:11.938130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:11.938235 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:11.938391 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:11.938793 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.940077 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.942105 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.942762 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.942868 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.943613 (+ 745us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:11.944404 (+ 791us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:11.944521 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:11.944784 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 04:23:11.944872 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:23:11.946262 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:11.937349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:39049 (local address 127.0.0.1:46350)
0807 04:23:11.937713 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:11.937735 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:11.937906 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:11.938433 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:11.938438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:11.938451 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:23:11.938643 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.938653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.940259 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.940265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.941938 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:11.941955 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:11.943110 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:11.943116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:11.943321 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:11.943392 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:23:11.944631 (+ 1239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:23:11.944646 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 04:23:11.945782 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0807 04:23:11.945950 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260807 04:23:11.951128 10072 raft_consensus.cc:1261] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Refusing update from remote peer 896594360f6e49c8af14a815c55bc4bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:11.951179 10215 raft_consensus.cc:1261] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Refusing update from remote peer 896594360f6e49c8af14a815c55bc4bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:11.952064 10301 consensus_queue.cc:1035] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:11.961056 10301 consensus_queue.cc:1035] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:11.967345 10304 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: TxnStatusTablet state changed. Reason: New leader 896594360f6e49c8af14a815c55bc4bd. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.967727 10304 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:11.972368 10301 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.972731 10301 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:23:11.979130 10304 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:11.979431 10304 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P b0a5bf6e41144c80816cbab240e38b71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:12.010070 10300 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: TxnStatusTablet state changed. Reason: New leader 896594360f6e49c8af14a815c55bc4bd. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:12.010430 10300 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:12.016191 10300 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:12.016458 10300 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 7110402f92f04b17afb8d64ddb5baf09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 04:23:12.019807 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.005401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59139 (local address 127.1.244.193:39049)
0807 04:23:12.005647 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.005656 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.012542 (+ 6886us) server_negotiation.cc:407] Connection header received
0807 04:23:12.012847 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.012852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.012973 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.013053 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:12.013924 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.015046 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.017105 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.017733 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.017831 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.018712 (+ 881us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.018899 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.018987 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.019250 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.019364 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:23:12.021121 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.005422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:39049 (local address 127.1.244.195:59139)
0807 04:23:12.012426 (+ 7004us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:12.012449 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:12.012674 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:12.013259 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:12.013264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:12.013278 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:23:12.013676 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.013715 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.015229 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.015234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.016940 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.016956 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.018064 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.018073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.018328 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.018427 (+ 99us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.019091 (+ 664us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.019106 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 04:23:12.020348 (+ 1242us) client_negotiation.cc:240] Negotiation successful
0807 04:23:12.020573 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6812,"thread_start_us":10487,"threads_started":1}
I20260807 04:23:12.028402 10347 tablet_replica.cc:439] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "896594360f6e49c8af14a815c55bc4bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } } }
I20260807 04:23:12.028921 10347 tablet_replica.cc:442] T 0627bb59c3884d1c8e8f36a42c87104e P 896594360f6e49c8af14a815c55bc4bd: This TxnStatusTablet replica's current role is: LEADER
I20260807 04:23:12.037230 9931 raft_consensus.cc:1261] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Refusing update from remote peer b0a5bf6e41144c80816cbab240e38b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:12.037930 10299 consensus_queue.cc:1035] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:12.040127 10072 raft_consensus.cc:1261] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Refusing update from remote peer b0a5bf6e41144c80816cbab240e38b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:12.040717 10299 consensus_queue.cc:1035] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:12.064065 10307 mvcc.cc:205] Tried to move back new op lower bound from 7315769720976330752 to 7315769719712501760. Current Snapshot: MvccSnapshot[applied={T|T < 7315769720976330752}]
I20260807 04:23:12.119236 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.100404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:38977 (local address 127.1.244.195:41151)
0807 04:23:12.100612 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.100619 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.103784 (+ 3165us) server_negotiation.cc:407] Connection header received
0807 04:23:12.112346 (+ 8562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.112353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.112463 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.112529 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:12.113364 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.114590 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.116872 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.117572 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.117669 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.118420 (+ 751us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.118548 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.118608 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.118835 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.118922 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:23:12.120072 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.101663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41151 (local address 127.1.244.193:38977)
0807 04:23:12.103651 (+ 1988us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:12.103696 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:12.103957 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:12.112725 (+ 8768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:12.112730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:12.112743 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:23:12.113146 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.113159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.114769 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.114775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.116709 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.116725 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.117920 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.117927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.118115 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.118188 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.118697 (+ 509us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.118707 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:23:12.119490 (+ 783us) client_negotiation.cc:240] Negotiation successful
0807 04:23:12.119644 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1734,"thread_start_us":1547,"threads_started":1}
I20260807 04:23:12.161689 9766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:12.162005 9766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:12.166000 9766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Sending DeleteTablet for 3 replicas of tablet c0b0178b7b1848fc8d7ed418e88ed855
I20260807 04:23:12.167213 9908 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b0178b7b1848fc8d7ed418e88ed855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46342
I20260807 04:23:12.167631 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b0178b7b1848fc8d7ed418e88ed855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40418
I20260807 04:23:12.167932 10318 tablet_replica.cc:330] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: stopping tablet replica
I20260807 04:23:12.167953 10356 tablet_replica.cc:330] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: stopping tablet replica
I20260807 04:23:12.168169 10318 raft_consensus.cc:2229] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:12.168162 10356 raft_consensus.cc:2229] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:12.168371 10318 raft_consensus.cc:2258] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.168421 10356 raft_consensus.cc:2258] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.168562 10194 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b0178b7b1848fc8d7ed418e88ed855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260807 04:23:12.170279 10318 ts_tablet_manager.cc:1905] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.179417 10318 ts_tablet_manager.cc:1918] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:23:12.180761 10356 ts_tablet_manager.cc:1905] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.188110 10318 log.cc:1196] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/c0b0178b7b1848fc8d7ed418e88ed855
I20260807 04:23:12.188908 10318 ts_tablet_manager.cc:1939] T c0b0178b7b1848fc8d7ed418e88ed855 P 7110402f92f04b17afb8d64ddb5baf09: Deleting consensus metadata
I20260807 04:23:12.190182 9750 catalog_manager.cc:4841] TS 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549): tablet c0b0178b7b1848fc8d7ed418e88ed855 (table test-table [id=325803a030cc4b0cb0050388ee960d9f]) successfully deleted
I20260807 04:23:12.190883 10357 tablet_replica.cc:330] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: stopping tablet replica
I20260807 04:23:12.191147 10357 raft_consensus.cc:2229] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:23:12.191586 10357 raft_consensus.cc:2258] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.192777 10357 ts_tablet_manager.cc:1905] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.201166 10357 ts_tablet_manager.cc:1918] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:23:12.201385 10357 log.cc:1196] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/c0b0178b7b1848fc8d7ed418e88ed855
I20260807 04:23:12.202031 10357 ts_tablet_manager.cc:1939] T c0b0178b7b1848fc8d7ed418e88ed855 P b0a5bf6e41144c80816cbab240e38b71: Deleting consensus metadata
I20260807 04:23:12.202155 10356 ts_tablet_manager.cc:1918] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:23:12.202397 10356 log.cc:1196] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/c0b0178b7b1848fc8d7ed418e88ed855
I20260807 04:23:12.203024 10356 ts_tablet_manager.cc:1939] T c0b0178b7b1848fc8d7ed418e88ed855 P 896594360f6e49c8af14a815c55bc4bd: Deleting consensus metadata
I20260807 04:23:12.203655 9751 catalog_manager.cc:4841] TS b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151): tablet c0b0178b7b1848fc8d7ed418e88ed855 (table test-table [id=325803a030cc4b0cb0050388ee960d9f]) successfully deleted
I20260807 04:23:12.204456 9750 catalog_manager.cc:4841] TS 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049): tablet c0b0178b7b1848fc8d7ed418e88ed855 (table test-table [id=325803a030cc4b0cb0050388ee960d9f]) successfully deleted
Aug 07 04:23:12 dist-test-slave-zkpd krb5kdc[9721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 04:23:12 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:12 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:23:12.255200 10330 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:12.235739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.1.244.254:46539)
0807 04:23:12.235995 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.236004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.236572 (+ 568us) server_negotiation.cc:407] Connection header received
0807 04:23:12.236895 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.236901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.236990 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.237063 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:12.239096 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.240414 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.242387 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.243191 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.243327 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.254910 (+ 11583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59390: BlockingRecv error: recv got EOF from 127.0.0.1:59390 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 04:23:12 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM
I20260807 04:23:12.279286 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.259428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.1.244.254:46539)
0807 04:23:12.259601 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.259607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.260097 (+ 490us) server_negotiation.cc:407] Connection header received
0807 04:23:12.260369 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.260376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.260457 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.260528 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:12.262506 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.263758 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.265780 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.266482 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.266619 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.271932 (+ 5313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:12.271939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:12.271944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:12.271985 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:12.275689 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:12.276575 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:12.276582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:12.276586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:12.276758 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:12.277462 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:12.277468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:12.277472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:12.277908 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:12.278250 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.278616 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.278874 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:23:12.283663 9766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:23:12.284382 9766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:12.291285 9908 tablet_service.cc:1467] Processing CreateTablet for tablet e00a1e52b42e442098bdc67c2c009799 (DEFAULT_TABLE table=test-table [id=4eda878a0fdd471a8dc4621c5578385c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:12.291754 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00a1e52b42e442098bdc67c2c009799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:12.291700 10194 tablet_service.cc:1467] Processing CreateTablet for tablet e00a1e52b42e442098bdc67c2c009799 (DEFAULT_TABLE table=test-table [id=4eda878a0fdd471a8dc4621c5578385c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:12.292114 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00a1e52b42e442098bdc67c2c009799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:12.292268 10050 tablet_service.cc:1467] Processing CreateTablet for tablet e00a1e52b42e442098bdc67c2c009799 (DEFAULT_TABLE table=test-table [id=4eda878a0fdd471a8dc4621c5578385c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:12.292734 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e00a1e52b42e442098bdc67c2c009799. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:12.296801 10294 tablet_bootstrap.cc:492] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Bootstrap starting.
I20260807 04:23:12.298038 10294 tablet_bootstrap.cc:654] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:12.300050 10293 tablet_bootstrap.cc:492] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Bootstrap starting.
I20260807 04:23:12.300441 10294 tablet_bootstrap.cc:492] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: No bootstrap required, opened a new log
I20260807 04:23:12.300570 10294 ts_tablet_manager.cc:1397] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260807 04:23:12.301367 10294 raft_consensus.cc:348] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.301636 10294 raft_consensus.cc:374] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:12.301683 10293 tablet_bootstrap.cc:654] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:12.301738 10294 raft_consensus.cc:729] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Initialized, Role: FOLLOWER
I20260807 04:23:12.301976 10294 consensus_queue.cc:260] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.302595 10294 ts_tablet_manager.cc:1428] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:23:12.306564 10296 tablet_bootstrap.cc:492] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Bootstrap starting.
I20260807 04:23:12.308552 10296 tablet_bootstrap.cc:654] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:12.309028 10293 tablet_bootstrap.cc:492] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: No bootstrap required, opened a new log
I20260807 04:23:12.309194 10293 ts_tablet_manager.cc:1397] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 04:23:12.310066 10293 raft_consensus.cc:348] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.310274 10293 raft_consensus.cc:374] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:12.310372 10293 raft_consensus.cc:729] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7110402f92f04b17afb8d64ddb5baf09, State: Initialized, Role: FOLLOWER
I20260807 04:23:12.310622 10293 consensus_queue.cc:260] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.311225 10293 ts_tablet_manager.cc:1428] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 04:23:12.311270 10296 tablet_bootstrap.cc:492] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: No bootstrap required, opened a new log
I20260807 04:23:12.311415 10296 ts_tablet_manager.cc:1397] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 04:23:12.312506 10296 raft_consensus.cc:348] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.312778 10296 raft_consensus.cc:374] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:12.312880 10296 raft_consensus.cc:729] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896594360f6e49c8af14a815c55bc4bd, State: Initialized, Role: FOLLOWER
I20260807 04:23:12.313359 10296 consensus_queue.cc:260] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.313984 10296 ts_tablet_manager.cc:1428] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 04:23:12.459098 10299 raft_consensus.cc:482] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:12.459302 10299 raft_consensus.cc:504] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.460032 10299 leader_election.cc:288] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049), 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549)
I20260807 04:23:12.460562 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00a1e52b42e442098bdc67c2c009799" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd" is_pre_election: true
I20260807 04:23:12.460637 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00a1e52b42e442098bdc67c2c009799" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09" is_pre_election: true
I20260807 04:23:12.460824 9931 raft_consensus.cc:2454] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 0.
I20260807 04:23:12.460863 10072 raft_consensus.cc:2454] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 0.
I20260807 04:23:12.461272 10144 leader_election.cc:302] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:12.461540 10299 raft_consensus.cc:2781] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:12.461668 10299 raft_consensus.cc:482] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:12.461768 10299 raft_consensus.cc:3037] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:12.463800 10299 raft_consensus.cc:504] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.464350 10299 leader_election.cc:288] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 election: Requested vote from peers 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049), 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549)
I20260807 04:23:12.464826 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00a1e52b42e442098bdc67c2c009799" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd"
I20260807 04:23:12.464901 10072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e00a1e52b42e442098bdc67c2c009799" candidate_uuid: "b0a5bf6e41144c80816cbab240e38b71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7110402f92f04b17afb8d64ddb5baf09"
I20260807 04:23:12.465018 9931 raft_consensus.cc:3037] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:12.465140 10072 raft_consensus.cc:3037] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:12.467172 9931 raft_consensus.cc:2454] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 1.
I20260807 04:23:12.467278 10072 raft_consensus.cc:2454] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a5bf6e41144c80816cbab240e38b71 in term 1.
I20260807 04:23:12.467619 10144 leader_election.cc:302] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 896594360f6e49c8af14a815c55bc4bd, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:12.467958 10299 raft_consensus.cc:2781] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:12.468119 10299 raft_consensus.cc:686] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 1 LEADER]: Becoming Leader. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Running, Role: LEADER
I20260807 04:23:12.468380 10299 consensus_queue.cc:237] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.470955 9765 catalog_manager.cc:5495] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "b0a5bf6e41144c80816cbab240e38b71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } health_report { overall_health: HEALTHY } } }
I20260807 04:23:12.502610 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.497904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.1.244.195:41151)
0807 04:23:12.498115 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.498120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.498243 (+ 123us) server_negotiation.cc:407] Connection header received
0807 04:23:12.498414 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.498417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.498483 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.498556 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:12.498985 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.499692 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.500931 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.501397 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.501492 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.501914 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.502013 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.502120 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.502235 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.502294 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:23:12.505185 9931 raft_consensus.cc:1261] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Refusing update from remote peer b0a5bf6e41144c80816cbab240e38b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:12.505203 10072 raft_consensus.cc:1261] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Refusing update from remote peer b0a5bf6e41144c80816cbab240e38b71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:12.505882 10312 consensus_queue.cc:1035] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:12.506191 10299 consensus_queue.cc:1035] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:12.512849 10308 mvcc.cc:205] Tried to move back new op lower bound from 7315769722895826944 to 7315769722752937984. Current Snapshot: MvccSnapshot[applied={T|T < 7315769722895826944}]
I20260807 04:23:12.517609 9764 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:12.517848 9764 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:12.520686 9764 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Sending DeleteTablet for 3 replicas of tablet e00a1e52b42e442098bdc67c2c009799
I20260807 04:23:12.521427 9908 tablet_service.cc:1514] Processing DeleteTablet for tablet e00a1e52b42e442098bdc67c2c009799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46342
I20260807 04:23:12.521644 10194 tablet_service.cc:1514] Processing DeleteTablet for tablet e00a1e52b42e442098bdc67c2c009799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260807 04:23:12.521735 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet e00a1e52b42e442098bdc67c2c009799 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40418
I20260807 04:23:12.522012 10356 tablet_replica.cc:330] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: stopping tablet replica
I20260807 04:23:12.522233 10356 raft_consensus.cc:2229] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:12.522468 10356 raft_consensus.cc:2258] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.522464 9764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 04:23:12.522738 10357 tablet_replica.cc:330] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: stopping tablet replica
I20260807 04:23:12.522939 10357 raft_consensus.cc:2229] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 1 LEADER]: Raft consensus shutting down.
W20260807 04:23:12.523056 9764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:12.523237 10357 raft_consensus.cc:2258] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.523423 10356 ts_tablet_manager.cc:1905] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.523535 10318 tablet_replica.cc:330] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: stopping tablet replica
I20260807 04:23:12.523689 10318 raft_consensus.cc:2229] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:12.523939 10318 raft_consensus.cc:2258] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.525035 10318 ts_tablet_manager.cc:1905] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.529358 9908 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff1cec2512c41c6bc15da2ed9792121 (DEFAULT_TABLE table=test-table [id=19666b2121ed44b4b2c3e286a18e7ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:12.529801 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff1cec2512c41c6bc15da2ed9792121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:12.529737 10194 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff1cec2512c41c6bc15da2ed9792121 (DEFAULT_TABLE table=test-table [id=19666b2121ed44b4b2c3e286a18e7ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:12.529935 10356 ts_tablet_manager.cc:1918] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:12.530064 10356 log.cc:1196] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/e00a1e52b42e442098bdc67c2c009799
I20260807 04:23:12.530182 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff1cec2512c41c6bc15da2ed9792121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:12.530510 10356 ts_tablet_manager.cc:1939] T e00a1e52b42e442098bdc67c2c009799 P 896594360f6e49c8af14a815c55bc4bd: Deleting consensus metadata
I20260807 04:23:12.531610 9750 catalog_manager.cc:4841] TS 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049): tablet e00a1e52b42e442098bdc67c2c009799 (table test-table [id=4eda878a0fdd471a8dc4621c5578385c]) successfully deleted
I20260807 04:23:12.532296 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff1cec2512c41c6bc15da2ed9792121 (DEFAULT_TABLE table=test-table [id=19666b2121ed44b4b2c3e286a18e7ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:12.532588 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff1cec2512c41c6bc15da2ed9792121. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:12.534721 10294 tablet_bootstrap.cc:492] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Bootstrap starting.
I20260807 04:23:12.535462 10357 ts_tablet_manager.cc:1905] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.536504 10294 tablet_bootstrap.cc:654] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:12.537927 10293 tablet_bootstrap.cc:492] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Bootstrap starting.
I20260807 04:23:12.540017 10293 tablet_bootstrap.cc:654] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:12.541429 10357 ts_tablet_manager.cc:1918] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:12.541584 10357 log.cc:1196] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/e00a1e52b42e442098bdc67c2c009799
I20260807 04:23:12.545356 10296 tablet_bootstrap.cc:492] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Bootstrap starting.
I20260807 04:23:12.545327 10318 ts_tablet_manager.cc:1918] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:12.545469 10318 log.cc:1196] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/e00a1e52b42e442098bdc67c2c009799
I20260807 04:23:12.545902 10318 ts_tablet_manager.cc:1939] T e00a1e52b42e442098bdc67c2c009799 P 7110402f92f04b17afb8d64ddb5baf09: Deleting consensus metadata
I20260807 04:23:12.541991 10357 ts_tablet_manager.cc:1939] T e00a1e52b42e442098bdc67c2c009799 P b0a5bf6e41144c80816cbab240e38b71: Deleting consensus metadata
I20260807 04:23:12.546967 10296 tablet_bootstrap.cc:654] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:12.547823 9751 catalog_manager.cc:4841] TS b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151): tablet e00a1e52b42e442098bdc67c2c009799 (table test-table [id=4eda878a0fdd471a8dc4621c5578385c]) successfully deleted
I20260807 04:23:12.548154 9750 catalog_manager.cc:4841] TS 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549): tablet e00a1e52b42e442098bdc67c2c009799 (table test-table [id=4eda878a0fdd471a8dc4621c5578385c]) successfully deleted
I20260807 04:23:12.550020 10293 tablet_bootstrap.cc:492] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: No bootstrap required, opened a new log
I20260807 04:23:12.550204 10293 ts_tablet_manager.cc:1397] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.011s
I20260807 04:23:12.550633 10296 tablet_bootstrap.cc:492] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: No bootstrap required, opened a new log
I20260807 04:23:12.550829 10296 ts_tablet_manager.cc:1397] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 04:23:12.550940 10294 tablet_bootstrap.cc:492] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: No bootstrap required, opened a new log
I20260807 04:23:12.551121 10294 ts_tablet_manager.cc:1397] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20260807 04:23:12.551049 10293 raft_consensus.cc:348] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.551314 10293 raft_consensus.cc:374] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:12.551399 10293 raft_consensus.cc:729] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7110402f92f04b17afb8d64ddb5baf09, State: Initialized, Role: FOLLOWER
I20260807 04:23:12.551654 10293 consensus_queue.cc:260] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.551637 10296 raft_consensus.cc:348] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.552016 10296 raft_consensus.cc:374] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:12.552146 10296 raft_consensus.cc:729] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 896594360f6e49c8af14a815c55bc4bd, State: Initialized, Role: FOLLOWER
I20260807 04:23:12.552273 10293 ts_tablet_manager.cc:1428] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 04:23:12.552417 10296 consensus_queue.cc:260] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.551882 10294 raft_consensus.cc:348] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.552768 10294 raft_consensus.cc:374] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:12.552896 10294 raft_consensus.cc:729] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a5bf6e41144c80816cbab240e38b71, State: Initialized, Role: FOLLOWER
I20260807 04:23:12.553171 10294 consensus_queue.cc:260] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.553788 10296 ts_tablet_manager.cc:1428] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 04:23:12.554265 10294 ts_tablet_manager.cc:1428] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:12.616374 10300 raft_consensus.cc:482] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:23:12.616652 10300 raft_consensus.cc:504] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.618492 10300 leader_election.cc:288] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049), b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151)
I20260807 04:23:12.627224 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.618979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45263 (local address 127.1.244.195:41151)
0807 04:23:12.619188 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.619194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.619836 (+ 642us) server_negotiation.cc:407] Connection header received
0807 04:23:12.620066 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.620070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.620129 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.620223 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:12.621053 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.622083 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.624821 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.626059 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.626227 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.626307 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.626824 (+ 517us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.626968 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:23:12.627259 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.619015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:58053 (local address 127.1.244.193:39049)
0807 04:23:12.619838 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.619842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.619856 (+ 14us) server_negotiation.cc:407] Connection header received
0807 04:23:12.619984 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.619987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.620054 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.620165 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:12.620824 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.621969 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.624825 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.626059 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.626227 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.626307 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.626835 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.626982 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":623,"threads_started":1}
I20260807 04:23:12.627532 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.618625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:39049 (local address 127.1.244.194:58053)
0807 04:23:12.619303 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:12.619331 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:12.619491 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:12.620195 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:12.620200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:12.620227 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:12.620625 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.620636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.622103 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.622107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.624652 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.624667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.626314 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.626319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.626597 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.626683 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 04:23:12.626781 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 04:23:12.626983 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":282,"thread_start_us":502,"threads_started":1}
I20260807 04:23:12.627548 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.618873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41151 (local address 127.1.244.194:45263)
0807 04:23:12.619717 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:12.619737 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:12.619916 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:12.620253 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:12.620258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:12.620285 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:12.620865 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.620881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.622218 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.622225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.624660 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.624669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.626276 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.626283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.626606 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.626690 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 04:23:12.626781 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 04:23:12.626983 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":252,"thread_start_us":471,"threads_started":1}
I20260807 04:23:12.628314 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff1cec2512c41c6bc15da2ed9792121" candidate_uuid: "7110402f92f04b17afb8d64ddb5baf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a5bf6e41144c80816cbab240e38b71" is_pre_election: true
I20260807 04:23:12.628337 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff1cec2512c41c6bc15da2ed9792121" candidate_uuid: "7110402f92f04b17afb8d64ddb5baf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd" is_pre_election: true
I20260807 04:23:12.628589 9931 raft_consensus.cc:2454] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7110402f92f04b17afb8d64ddb5baf09 in term 0.
I20260807 04:23:12.628568 10215 raft_consensus.cc:2454] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7110402f92f04b17afb8d64ddb5baf09 in term 0.
I20260807 04:23:12.629194 10002 leader_election.cc:302] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, 896594360f6e49c8af14a815c55bc4bd, b0a5bf6e41144c80816cbab240e38b71; no voters:
I20260807 04:23:12.629616 10300 raft_consensus.cc:2781] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:12.629761 10300 raft_consensus.cc:482] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:12.629865 10300 raft_consensus.cc:3037] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:12.632113 10300 raft_consensus.cc:504] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.632793 10300 leader_election.cc:288] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [CANDIDATE]: Term 1 election: Requested vote from peers 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049), b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151)
I20260807 04:23:12.633219 9931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff1cec2512c41c6bc15da2ed9792121" candidate_uuid: "7110402f92f04b17afb8d64ddb5baf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "896594360f6e49c8af14a815c55bc4bd"
I20260807 04:23:12.633462 9931 raft_consensus.cc:3037] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:12.633447 10215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff1cec2512c41c6bc15da2ed9792121" candidate_uuid: "7110402f92f04b17afb8d64ddb5baf09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a5bf6e41144c80816cbab240e38b71"
I20260807 04:23:12.633599 10215 raft_consensus.cc:3037] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:12.635417 9931 raft_consensus.cc:2454] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7110402f92f04b17afb8d64ddb5baf09 in term 1.
I20260807 04:23:12.635478 10215 raft_consensus.cc:2454] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7110402f92f04b17afb8d64ddb5baf09 in term 1.
I20260807 04:23:12.635814 10001 leader_election.cc:302] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7110402f92f04b17afb8d64ddb5baf09, 896594360f6e49c8af14a815c55bc4bd; no voters:
I20260807 04:23:12.636145 10300 raft_consensus.cc:2781] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:12.636540 10300 raft_consensus.cc:686] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 LEADER]: Becoming Leader. State: Replica: 7110402f92f04b17afb8d64ddb5baf09, State: Running, Role: LEADER
I20260807 04:23:12.636883 10300 consensus_queue.cc:237] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } }
I20260807 04:23:12.640256 9766 catalog_manager.cc:5495] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "7110402f92f04b17afb8d64ddb5baf09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7110402f92f04b17afb8d64ddb5baf09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 } health_report { overall_health: UNKNOWN } } }
I20260807 04:23:12.679415 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.673821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.1.244.194:43549)
0807 04:23:12.674345 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.674349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.674367 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:23:12.674444 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.674447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.674518 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.674651 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:12.674992 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.675741 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.677141 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.677590 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.677730 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.678193 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.678767 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.678878 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.678993 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.679052 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":298,"threads_started":1}
I20260807 04:23:12.693035 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.688249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43973 (local address 127.1.244.193:39049)
0807 04:23:12.688477 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.688484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.688875 (+ 391us) server_negotiation.cc:407] Connection header received
0807 04:23:12.688993 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.688997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.689057 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.689125 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:12.689537 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.690205 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.691482 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.691846 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.692039 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.692510 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.692596 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.692660 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.692794 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.692850 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:23:12.693166 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.688179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:39049 (local address 127.1.244.194:43973)
0807 04:23:12.688772 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:12.688793 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:12.688906 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:12.689151 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:12.689154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:12.689163 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:23:12.689395 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.689402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.690319 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.690322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.691359 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.691370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.692093 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.692097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.692258 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.692333 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.692683 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.692693 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:23:12.692757 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 04:23:12.692848 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":283,"threads_started":1}
I20260807 04:23:12.700598 10215 raft_consensus.cc:1261] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Refusing update from remote peer 7110402f92f04b17afb8d64ddb5baf09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:12.700645 9931 raft_consensus.cc:1261] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Refusing update from remote peer 7110402f92f04b17afb8d64ddb5baf09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:12.701247 10383 consensus_queue.cc:1035] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a5bf6e41144c80816cbab240e38b71" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:12.701594 10300 consensus_queue.cc:1035] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "896594360f6e49c8af14a815c55bc4bd" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:12.726562 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.721321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51229 (local address 127.1.244.194:43549)
0807 04:23:12.721555 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.721563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.722126 (+ 563us) server_negotiation.cc:407] Connection header received
0807 04:23:12.722180 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.722183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.722240 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.722304 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:12.722659 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.723353 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.724998 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.725593 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.725690 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.726031 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.726122 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.726186 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.726308 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.726369 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:23:12.726603 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.721269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:43549 (local address 127.1.244.193:51229)
0807 04:23:12.721824 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:12.721841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:12.721997 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:12.722346 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:12.722349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:12.722359 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 04:23:12.722562 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.722570 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.723491 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.723495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.724846 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:12.724859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.725728 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.725732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:12.725854 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.725898 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 04:23:12.726203 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 04:23:12.726211 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:23:12.726268 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 04:23:12.726372 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":281,"threads_started":1}
I20260807 04:23:12.753710 9766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:12.753985 9766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:12.757056 9766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc7a009d61d149cfbbab2f0887497bde: Sending DeleteTablet for 3 replicas of tablet 7ff1cec2512c41c6bc15da2ed9792121
I20260807 04:23:12.757848 9908 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff1cec2512c41c6bc15da2ed9792121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:46342
I20260807 04:23:12.757911 10194 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff1cec2512c41c6bc15da2ed9792121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35386
I20260807 04:23:12.757907 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff1cec2512c41c6bc15da2ed9792121 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:12 UTC) from {username='oryx', principal='oryx/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40418
I20260807 04:23:12.758157 10356 tablet_replica.cc:330] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: stopping tablet replica
I20260807 04:23:12.758175 10357 tablet_replica.cc:330] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: stopping tablet replica
I20260807 04:23:12.758296 10357 raft_consensus.cc:2229] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:12.758323 10356 raft_consensus.cc:2229] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:12.758441 10357 raft_consensus.cc:2258] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.758515 10356 raft_consensus.cc:2258] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.758713 10318 tablet_replica.cc:330] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: stopping tablet replica
I20260807 04:23:12.758949 10318 raft_consensus.cc:2229] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:23:12.759413 10318 raft_consensus.cc:2258] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:12.760035 10356 ts_tablet_manager.cc:1905] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.760435 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 9841
I20260807 04:23:12.760680 10318 ts_tablet_manager.cc:1905] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.766956 10356 ts_tablet_manager.cc:1918] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:23:12.767166 10356 log.cc:1196] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/7ff1cec2512c41c6bc15da2ed9792121
I20260807 04:23:12.767761 10356 ts_tablet_manager.cc:1939] T 7ff1cec2512c41c6bc15da2ed9792121 P 896594360f6e49c8af14a815c55bc4bd: Deleting consensus metadata
I20260807 04:23:12.769070 9750 catalog_manager.cc:4841] TS 896594360f6e49c8af14a815c55bc4bd (127.1.244.193:39049): tablet 7ff1cec2512c41c6bc15da2ed9792121 (table test-table [id=19666b2121ed44b4b2c3e286a18e7ce6]) successfully deleted
I20260807 04:23:12.769964 10357 ts_tablet_manager.cc:1905] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:12.775209 10318 ts_tablet_manager.cc:1918] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:23:12.775444 10318 log.cc:1196] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/7ff1cec2512c41c6bc15da2ed9792121
I20260807 04:23:12.776012 10318 ts_tablet_manager.cc:1939] T 7ff1cec2512c41c6bc15da2ed9792121 P 7110402f92f04b17afb8d64ddb5baf09: Deleting consensus metadata
I20260807 04:23:12.776167 10357 ts_tablet_manager.cc:1918] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 04:23:12.776329 10357 log.cc:1196] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/7ff1cec2512c41c6bc15da2ed9792121
I20260807 04:23:12.776901 10357 ts_tablet_manager.cc:1939] T 7ff1cec2512c41c6bc15da2ed9792121 P b0a5bf6e41144c80816cbab240e38b71: Deleting consensus metadata
I20260807 04:23:12.778141 9750 catalog_manager.cc:4841] TS 7110402f92f04b17afb8d64ddb5baf09 (127.1.244.194:43549): tablet 7ff1cec2512c41c6bc15da2ed9792121 (table test-table [id=19666b2121ed44b4b2c3e286a18e7ce6]) successfully deleted
I20260807 04:23:12.778687 9751 catalog_manager.cc:4841] TS b0a5bf6e41144c80816cbab240e38b71 (127.1.244.195:41151): tablet 7ff1cec2512c41c6bc15da2ed9792121 (table test-table [id=19666b2121ed44b4b2c3e286a18e7ce6]) successfully deleted
Aug 07 04:23:12 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.193@KRBTEST.COM
I20260807 04:23:12.786827 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.765047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.1.244.193:39049)
0807 04:23:12.765311 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.765320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.769781 (+ 4461us) server_negotiation.cc:407] Connection header received
0807 04:23:12.770058 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.770063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.770148 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.770279 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:12.772013 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.773138 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.774992 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.775595 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.775687 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.781230 (+ 5543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:12.781243 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:12.781258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:12.781288 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:12.784172 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:12.784947 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:12.784954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:12.784956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:12.785076 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:12.785475 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:12.785478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:12.785480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:12.785828 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:12.786043 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.786377 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.786531 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 04:23:12.787513 9975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:12.913264 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 9841
I20260807 04:23:12.945866 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 9984
Aug 07 04:23:12 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.194@KRBTEST.COM
I20260807 04:23:12.963445 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:12.946721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.1.244.194:43549)
0807 04:23:12.946930 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:12.946935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:12.946977 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:23:12.947187 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:12.947193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:12.947274 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:12.947389 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:12.948799 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.949979 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.951193 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:12.951919 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:12.952051 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:12.956515 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:12.956530 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:12.956549 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:12.956589 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:12.960729 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:12.961394 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:12.961403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:12.961407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:12.961559 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:12.961920 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:12.961925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:12.961928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:12.962353 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:12.962717 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:12.962854 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 04:23:12.963072 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:23:12.964164 10118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:13.087837 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 9984
I20260807 04:23:13.110592 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 10127
Aug 07 04:23:13 dist-test-slave-zkpd krb5kdc[9721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.244.195@KRBTEST.COM
I20260807 04:23:13.125651 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:13.111310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.1.244.195:41151)
0807 04:23:13.111498 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:13.111504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:13.111600 (+ 96us) server_negotiation.cc:407] Connection header received
0807 04:23:13.111815 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:13.111819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:13.111937 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:13.112051 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:13.113284 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:13.114019 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:13.115247 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:13.115665 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:13.115798 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:13.120176 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:13.120191 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:13.120208 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:13.120238 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:13.122955 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:13.123623 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:13.123630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:13.123633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:13.123751 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:13.124163 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:13.124169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:13.124174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:13.124524 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:13.124873 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:13.125097 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 04:23:13.125334 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:23:13.126288 10261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:13.250169 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 10127
I20260807 04:23:13.274734 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 9734
I20260807 04:23:13.276039 9809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:13.394676 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 9734
2026-08-07T04:23:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6855 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:23:13 dist-test-slave-zkpd krb5kdc[10405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:23:13 dist-test-slave-zkpd krb5kdc[10405](info): set up 2 sockets
Aug 07 04:23:13 dist-test-slave-zkpd krb5kdc[10405](info): commencing operation
krb5kdc: starting...
W20260807 04:23:14.523821 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:23:14 dist-test-slave-zkpd krb5kdc[10405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:23:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:23:14.759130 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37777
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:15.087760 10419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:15.088104 10419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:15.088274 10419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:15.095311 10419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:15.095402 10419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:15.095445 10419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:15.095492 10419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:15.095531 10419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:15.105497 10419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:15.106608 10419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:15.107839 10419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:15.120555 10427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:15.120560 10425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:15.121132 10424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:15.121258 10419 server_base.cc:1034] running on GCE node
I20260807 04:23:15.122079 10419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:15.123448 10419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:15.124671 10419 hybrid_clock.cc:648] HybridClock initialized: now 1786076595124633 us; error 50 us; skew 500 ppm
I20260807 04:23:15.125015 10419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T04:23:15Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1798 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:23:15 dist-test-slave-zkpd krb5kdc[10433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:23:15 dist-test-slave-zkpd krb5kdc[10433](info): set up 2 sockets
Aug 07 04:23:15 dist-test-slave-zkpd krb5kdc[10433](info): commencing operation
krb5kdc: starting...
W20260807 04:23:16.299352 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:23:16 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:23:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:23:16.535495 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35625
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:16.859791 10446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:16.860075 10446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:16.860261 10446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:16.867342 10446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:16.867470 10446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:16.867570 10446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:16.867671 10446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:16.867733 10446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:16.877116 10446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:16.878333 10446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:16.879591 10446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:16.891359 10451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:16.891444 10452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:16.891402 10454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:16.892150 10446 server_base.cc:1034] running on GCE node
I20260807 04:23:16.892848 10446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:16.894146 10446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:16.895375 10446 hybrid_clock.cc:648] HybridClock initialized: now 1786076596895338 us; error 47 us; skew 500 ppm
I20260807 04:23:16.895712 10446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:16 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076596, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:16.898788 10446 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:23:16.900187 10446 webserver.cc:460] Webserver started at http://127.1.244.254:40813/ using document root <none> and password file <none>
I20260807 04:23:16.900836 10446 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:16.900983 10446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:16.901299 10446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:16.904222 10446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "79ce7382576e434687e013700ed8137a"
format_stamp: "Formatted at 2026-08-07 04:23:16 on dist-test-slave-zkpd"
server_key: "c9728c455a6f61530eb7ce443e6524e0"
server_key_iv: "d48cbf4037027088c85d3fc7bcc39ee9"
server_key_version: "encryptionkey@0"
I20260807 04:23:16.905025 10446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "79ce7382576e434687e013700ed8137a"
format_stamp: "Formatted at 2026-08-07 04:23:16 on dist-test-slave-zkpd"
server_key: "c9728c455a6f61530eb7ce443e6524e0"
server_key_iv: "d48cbf4037027088c85d3fc7bcc39ee9"
server_key_version: "encryptionkey@0"
I20260807 04:23:16.909832 10446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:23:16.913017 10461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:16.914176 10446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 04:23:16.914356 10446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "79ce7382576e434687e013700ed8137a"
format_stamp: "Formatted at 2026-08-07 04:23:16 on dist-test-slave-zkpd"
server_key: "c9728c455a6f61530eb7ce443e6524e0"
server_key_iv: "d48cbf4037027088c85d3fc7bcc39ee9"
server_key_version: "encryptionkey@0"
I20260807 04:23:16.914544 10446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:16.939154 10446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:16.942934 10446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:16.943202 10446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:16.961403 10446 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35625
I20260807 04:23:16.961428 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35625 every 8 connection(s)
I20260807 04:23:16.962754 10446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:23:16.966440 10516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:16.969699 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 10446
I20260807 04:23:16.969933 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:23:16.970768 2003 external_mini_cluster.cc:1428] Setting key e358a66f70454b79249de46e144f0eca
I20260807 04:23:16.979542 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Bootstrap starting.
I20260807 04:23:16.983367 10516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:16.984750 10516 log.cc:824] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:23:16 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:16.987977 10516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: No bootstrap required, opened a new log
I20260807 04:23:16.993887 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:16.972865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.1.244.254:35625)
0807 04:23:16.973607 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:16.973618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:16.973675 (+ 57us) server_negotiation.cc:407] Connection header received
0807 04:23:16.975324 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:16.975346 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:16.975734 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:16.976137 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:16.977936 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:16.979187 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:16.980927 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:16.981577 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:16.981719 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:16.986706 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:16.986714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:16.986727 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:16.986765 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:16.989918 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:16.990752 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:16.990758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:16.990766 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:16.990922 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:16.991292 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:16.991295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:16.991297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:16.991782 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:16.992062 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:16.992486 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 04:23:16.992886 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":344,"threads_started":1}
I20260807 04:23:16.997378 10516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:16.997741 10516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:16.997855 10516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ce7382576e434687e013700ed8137a, State: Initialized, Role: FOLLOWER
I20260807 04:23:16.998446 10516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:16.998720 10516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:23:16.998883 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:23:16.999078 10516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:17.001235 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:17.001714 10516 leader_election.cc:302] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ce7382576e434687e013700ed8137a; no voters:
I20260807 04:23:17.002607 10516 leader_election.cc:288] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:23:17.002727 10522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:17.003618 10522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 LEADER]: Becoming Leader. State: Replica: 79ce7382576e434687e013700ed8137a, State: Running, Role: LEADER
I20260807 04:23:17.004175 10522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:17.004717 10516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:17.006927 10524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ce7382576e434687e013700ed8137a. Latest consensus state: current_term: 1 leader_uuid: "79ce7382576e434687e013700ed8137a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } } }
I20260807 04:23:17.006863 10523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ce7382576e434687e013700ed8137a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } } }
I20260807 04:23:17.007310 10524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:17.007310 10523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:17.009642 10527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:17.015795 10527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:17.023795 10527 catalog_manager.cc:1347] Generated new cluster ID: 5db6a1058daa43aba78e3ad967f432e2
I20260807 04:23:17.023932 10527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:17.039819 10527 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:23:17.041136 10527 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:17.045885 10527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Generated new TSK 0
I20260807 04:23:17.046571 10527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:23:17.118974 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35625
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:17.469087 10543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:17.469398 10543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:17.469561 10543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:17.476681 10543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:17.476824 10543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:17.477018 10543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:17.486469 10543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:17.487694 10543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:17.489014 10543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:17.501885 10549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:17.501997 10552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:17.501997 10550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:17.502697 10543 server_base.cc:1034] running on GCE node
I20260807 04:23:17.503262 10543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:17.504209 10543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:17.505405 10543 hybrid_clock.cc:648] HybridClock initialized: now 1786076597505327 us; error 61 us; skew 500 ppm
I20260807 04:23:17.505769 10543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:17 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076597, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:17.512984 10543 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:23:17.514394 10543 webserver.cc:460] Webserver started at http://127.1.244.193:34907/ using document root <none> and password file <none>
I20260807 04:23:17.515014 10543 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:17.515115 10543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:17.515434 10543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:17.518321 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "41d5f83095bd43499b325472544494fd"
format_stamp: "Formatted at 2026-08-07 04:23:17 on dist-test-slave-zkpd"
server_key: "ffaaa74d9eebb282b09bdb34b947063b"
server_key_iv: "1823142eebb9ffe8a130512aeacf1d53"
server_key_version: "encryptionkey@0"
I20260807 04:23:17.519099 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "41d5f83095bd43499b325472544494fd"
format_stamp: "Formatted at 2026-08-07 04:23:17 on dist-test-slave-zkpd"
server_key: "ffaaa74d9eebb282b09bdb34b947063b"
server_key_iv: "1823142eebb9ffe8a130512aeacf1d53"
server_key_version: "encryptionkey@0"
I20260807 04:23:17.524163 10543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 04:23:17.527537 10559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:17.528877 10543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:17.529052 10543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "41d5f83095bd43499b325472544494fd"
format_stamp: "Formatted at 2026-08-07 04:23:17 on dist-test-slave-zkpd"
server_key: "ffaaa74d9eebb282b09bdb34b947063b"
server_key_iv: "1823142eebb9ffe8a130512aeacf1d53"
server_key_version: "encryptionkey@0"
I20260807 04:23:17.529254 10543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:17.550558 10543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:17.554500 10543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:17.554785 10543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:17.555730 10543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:17.557237 10543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:17.557315 10543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:17.557485 10543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:17.557544 10543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:17.590446 10543 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34291
I20260807 04:23:17.590535 10674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34291 every 8 connection(s)
I20260807 04:23:17.591746 10543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:17.597071 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 10543
I20260807 04:23:17.597285 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:23:17.598109 2003 external_mini_cluster.cc:1428] Setting key d5808d67b4c198a89ab1f11e936d2c11
Aug 07 04:23:17 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076597, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:17.614082 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:17.594345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:32855 (local address 127.1.244.254:35625)
0807 04:23:17.594805 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:17.594809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:17.595595 (+ 786us) server_negotiation.cc:407] Connection header received
0807 04:23:17.598038 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:17.598042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:17.598110 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:17.598257 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:17.600162 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:17.601248 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:17.602751 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:17.603476 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:17.603615 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:17.607140 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:17.607149 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:17.607156 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:17.607200 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:17.610768 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:17.611718 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:17.611722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:17.611725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:17.611838 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:17.612366 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:17.612369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:17.612371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:17.612619 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:17.612726 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:17.613579 (+ 853us) server_negotiation.cc:299] Negotiation successful
0807 04:23:17.613819 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":250,"threads_started":1}
I20260807 04:23:17.615568 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:17.594722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:32855)
0807 04:23:17.595425 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:17.595465 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:17.597715 (+ 2250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:17.598551 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:17.598562 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:17.599141 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:17.599967 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:17.600011 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:17.601410 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:17.601414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:17.602600 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:17.602612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:17.603671 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:17.603675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:17.603857 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:17.604580 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:17.604601 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:17.606908 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:17.610896 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:17.610907 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:17.610926 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:17.611604 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:17.612009 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:17.612016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:17.612023 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:17.612255 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:17.612739 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:17.612749 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:17.613194 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 04:23:17.613483 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 04:23:17.614108 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":271,"threads_started":1}
I20260807 04:23:17.617156 10675 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35625
I20260807 04:23:17.617473 10675 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:17.618332 10675 heartbeater.cc:511] Master 127.1.244.254:35625 requested a full tablet report, sending...
I20260807 04:23:17.620491 10478 ts_manager.cc:194] Registered new tserver with Master: 41d5f83095bd43499b325472544494fd (127.1.244.193:34291)
I20260807 04:23:17.621968 10478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:32855
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:23:17.654889 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35625
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:17.974669 10681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:17.974983 10681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:17.975147 10681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:17.982057 10681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:17.982200 10681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:17.982370 10681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:17.991539 10681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:17.992758 10681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:17.994020 10681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:18.006793 10686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:18.006976 10687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:18.006922 10689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:18.007809 10681 server_base.cc:1034] running on GCE node
I20260807 04:23:18.008512 10681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:18.009459 10681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:18.010663 10681 hybrid_clock.cc:648] HybridClock initialized: now 1786076598010614 us; error 37 us; skew 500 ppm
I20260807 04:23:18.011004 10681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:18 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:18.014148 10681 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:23:18.015650 10681 webserver.cc:460] Webserver started at http://127.1.244.194:34533/ using document root <none> and password file <none>
I20260807 04:23:18.016464 10681 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:18.016588 10681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:18.016919 10681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:18.019855 10681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "c8ea744ba9b14ea39a713d5e52111671"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "b7c6961b0431319ff6e2ba92de739ba3"
server_key_iv: "f815b5dcaf0ff3cfb746b5354d65a671"
server_key_version: "encryptionkey@0"
I20260807 04:23:18.020721 10681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "c8ea744ba9b14ea39a713d5e52111671"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "b7c6961b0431319ff6e2ba92de739ba3"
server_key_iv: "f815b5dcaf0ff3cfb746b5354d65a671"
server_key_version: "encryptionkey@0"
I20260807 04:23:18.025806 10681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:23:18.029191 10697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:18.030448 10681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:23:18.030611 10681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "c8ea744ba9b14ea39a713d5e52111671"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "b7c6961b0431319ff6e2ba92de739ba3"
server_key_iv: "f815b5dcaf0ff3cfb746b5354d65a671"
server_key_version: "encryptionkey@0"
I20260807 04:23:18.030807 10681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:18.054147 10681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:18.058490 10681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:18.058781 10681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:18.059734 10681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:18.061120 10681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:18.061193 10681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:18.061353 10681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:18.061408 10681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:18.093935 10681 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45083
I20260807 04:23:18.094019 10811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45083 every 8 connection(s)
I20260807 04:23:18.095184 10681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:18.096635 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 10681
I20260807 04:23:18.096818 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:23:18.097360 2003 external_mini_cluster.cc:1428] Setting key 9decbc312e1b1bb5dcc890b8f459b189
Aug 07 04:23:18 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:18.117938 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:18.097863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41799 (local address 127.1.244.254:35625)
0807 04:23:18.098075 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:18.098081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:18.099325 (+ 1244us) server_negotiation.cc:407] Connection header received
0807 04:23:18.102156 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:18.102160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:18.102220 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:18.102305 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:18.104311 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.105147 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.107560 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.108026 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.108120 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:18.112049 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:18.112054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:18.112057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:18.112087 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:18.114797 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.115534 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.115539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.115541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.115661 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.116175 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.116182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.116190 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.116459 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:18.116617 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:18.117359 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 04:23:18.117594 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:23:18.118939 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:18.098381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:41799)
0807 04:23:18.099163 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:18.099212 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:18.101866 (+ 2654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:18.102566 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:18.102598 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:18.103346 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:18.104148 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:18.104178 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.105319 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.105327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:18.107426 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:18.107444 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.108231 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.108240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:18.108553 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:18.109238 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:18.109271 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:18.111803 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:18.114962 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:18.114970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:18.114985 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:18.115401 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:18.115772 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:18.115777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:18.115780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:18.116054 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:18.116598 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:18.116606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:18.116928 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 04:23:18.117204 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 04:23:18.117724 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":296,"threads_started":1}
I20260807 04:23:18.120365 10812 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35625
I20260807 04:23:18.120685 10812 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:18.121393 10812 heartbeater.cc:511] Master 127.1.244.254:35625 requested a full tablet report, sending...
I20260807 04:23:18.122867 10478 ts_manager.cc:194] Registered new tserver with Master: c8ea744ba9b14ea39a713d5e52111671 (127.1.244.194:45083)
I20260807 04:23:18.123656 10478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41799
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:23:18.154861 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35625
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:18.480237 10817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:18.480494 10817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:18.480587 10817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:18.487349 10817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:18.487440 10817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:18.487545 10817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:18.496939 10817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:18.498203 10817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:18.499411 10817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:18.512238 10823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:18.512284 10822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:18.512550 10825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:18.512867 10817 server_base.cc:1034] running on GCE node
I20260807 04:23:18.513453 10817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:18.514353 10817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:18.515576 10817 hybrid_clock.cc:648] HybridClock initialized: now 1786076598515540 us; error 49 us; skew 500 ppm
I20260807 04:23:18.515883 10817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:18 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:18.519161 10817 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:23:18.520591 10817 webserver.cc:460] Webserver started at http://127.1.244.195:40251/ using document root <none> and password file <none>
I20260807 04:23:18.521256 10817 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:18.521399 10817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:18.521705 10817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:18.524580 10817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "23fb22f5bd8f4149a5a1a634dc43a482"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "ef4f3191ceca11a713de536521f4342d"
server_key_iv: "4f2cea9a9899a8e0f41dd7d710fc44b9"
server_key_version: "encryptionkey@0"
I20260807 04:23:18.525391 10817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "23fb22f5bd8f4149a5a1a634dc43a482"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "ef4f3191ceca11a713de536521f4342d"
server_key_iv: "4f2cea9a9899a8e0f41dd7d710fc44b9"
server_key_version: "encryptionkey@0"
I20260807 04:23:18.530499 10817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:23:18.534235 10832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:18.535626 10817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:23:18.535804 10817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "23fb22f5bd8f4149a5a1a634dc43a482"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "ef4f3191ceca11a713de536521f4342d"
server_key_iv: "4f2cea9a9899a8e0f41dd7d710fc44b9"
server_key_version: "encryptionkey@0"
I20260807 04:23:18.536067 10817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:18.547386 10817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:18.551553 10817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:18.551832 10817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:18.552727 10817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:18.553997 10817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:18.554101 10817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:18.554251 10817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:18.554327 10817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:18.587445 10817 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40459
I20260807 04:23:18.587487 10948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40459 every 8 connection(s)
I20260807 04:23:18.588785 10817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:18.596557 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 10817
I20260807 04:23:18.596756 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:23:18.597187 2003 external_mini_cluster.cc:1428] Setting key c5651bbbe4e03b8d39f4794f0bde1e07
Aug 07 04:23:18 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:18.609141 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:18.591493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54603 (local address 127.1.244.254:35625)
0807 04:23:18.591704 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:18.591711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:18.592980 (+ 1269us) server_negotiation.cc:407] Connection header received
0807 04:23:18.595506 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:18.595511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:18.595609 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:18.595691 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:18.597476 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.598410 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.600165 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.600580 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.600675 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:18.603978 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:18.603982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:18.603985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:18.604019 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:18.606560 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.607205 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.607209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.607211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.607333 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.607693 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.607696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.607698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.608020 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:18.608131 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:18.608732 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 04:23:18.608886 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:23:18.609997 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:18.591962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:54603)
0807 04:23:18.592774 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:18.592832 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:18.595194 (+ 2362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:18.595977 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:18.595989 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:18.596625 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:18.597301 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:18.597321 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.598560 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.598568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:18.600013 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:18.600026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.600764 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.600769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:18.600958 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:18.601636 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:18.601658 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:18.603735 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:18.606679 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:18.606685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:18.606696 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:18.607100 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:18.607422 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:18.607426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:18.607428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:18.607594 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:18.608128 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:18.608134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:18.608409 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 04:23:18.608630 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 04:23:18.608985 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":321,"threads_started":1}
I20260807 04:23:18.611565 10949 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35625
I20260807 04:23:18.611934 10949 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:18.612674 10949 heartbeater.cc:511] Master 127.1.244.254:35625 requested a full tablet report, sending...
I20260807 04:23:18.613941 10478 ts_manager.cc:194] Registered new tserver with Master: 23fb22f5bd8f4149a5a1a634dc43a482 (127.1.244.195:40459)
I20260807 04:23:18.614711 10478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:54603
I20260807 04:23:18.624466 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:23:18.624897 10675 heartbeater.cc:503] Master 127.1.244.254:35625 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
Entry for principal oryx/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab.
WARNING: no policy specified for oryx/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
Entry for principal oryx/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab.
WARNING: no policy specified for oryx/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
Entry for principal oryx/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab.
I20260807 04:23:18.766188 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 10543
Aug 07 04:23:18 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:23:18.782104 10959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:18.767404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.1.244.193:34291)
0807 04:23:18.768146 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:18.768153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:18.768180 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:23:18.768242 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:18.768249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:18.768468 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:18.768612 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:18.769919 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.770942 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.772163 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.772713 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.772811 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:18.776801 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:18.776813 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:18.776826 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:18.776857 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:18.779726 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.780440 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.780447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.780449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.780569 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.780885 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.780895 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.780897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.781219 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:18.781438 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:18.781638 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 04:23:18.781815 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":395,"threads_started":1}
I20260807 04:23:18.783355 10670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:18.865334 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 10543
I20260807 04:23:18.887303 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 10681
Aug 07 04:23:18 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:23:18.903103 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:18.888398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.1.244.194:45083)
0807 04:23:18.889550 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:18.889557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:18.889582 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:23:18.889643 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:18.889649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:18.889846 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:18.889978 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:18.891164 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.892316 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.893556 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:18.894124 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:18.894221 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:18.898188 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:18.898197 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:18.898210 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:18.898248 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:18.900877 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.901452 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.901458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.901460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.901570 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:18.901859 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:18.901862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:18.901864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:18.902163 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:18.902398 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:18.902626 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 04:23:18.902816 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":843,"threads_started":1}
I20260807 04:23:18.904361 10807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:18.985116 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 10681
I20260807 04:23:19.002451 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 10817
Aug 07 04:23:19 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:23:19.018949 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:19.003617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.1.244.195:40459)
0807 04:23:19.004320 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:19.004327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:19.004353 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:23:19.004416 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:19.004422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:19.004622 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:19.004743 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:19.006184 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:19.007200 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:19.008488 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:19.009056 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:19.009153 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:19.013501 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:19.013512 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:19.013526 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:19.013555 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:19.016363 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:19.016965 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:19.016971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:19.016973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:19.017120 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:19.017480 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:19.017483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:19.017485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:19.017793 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:19.018041 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:19.018292 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 04:23:19.018618 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":404,"threads_started":1}
I20260807 04:23:19.020210 10944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:19.101943 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 10817
I20260807 04:23:19.120028 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 10446
I20260807 04:23:19.121183 10510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:19.224593 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 10446
I20260807 04:23:19.248322 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35625
--webserver_interface=127.1.244.254
--webserver_port=40813
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:19.577286 10966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:19.577545 10966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:19.577696 10966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:19.584784 10966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:19.584883 10966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:19.584964 10966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:19.585080 10966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:19.585161 10966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:19.594611 10966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=40813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:19.595849 10966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:19.597157 10966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:19.609053 10971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:19.609086 10974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:19.609017 10972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:19.609871 10966 server_base.cc:1034] running on GCE node
I20260807 04:23:19.610589 10966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:19.611959 10966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:19.613169 10966 hybrid_clock.cc:648] HybridClock initialized: now 1786076599613112 us; error 50 us; skew 500 ppm
I20260807 04:23:19.613507 10966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:19 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076599, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:19.616595 10966 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:23:19.618038 10966 webserver.cc:460] Webserver started at http://127.1.244.254:40813/ using document root <none> and password file <none>
I20260807 04:23:19.618731 10966 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:19.618882 10966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:19.624313 10966 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:23:19.627099 10983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:19.628463 10966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:19.628646 10966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "79ce7382576e434687e013700ed8137a"
format_stamp: "Formatted at 2026-08-07 04:23:16 on dist-test-slave-zkpd"
server_key: "c9728c455a6f61530eb7ce443e6524e0"
server_key_iv: "d48cbf4037027088c85d3fc7bcc39ee9"
server_key_version: "encryptionkey@0"
I20260807 04:23:19.629601 10966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:19.648531 10966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:19.652251 10966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:19.652504 10966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:19.671075 10966 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35625
I20260807 04:23:19.671093 11036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35625 every 8 connection(s)
I20260807 04:23:19.672559 10966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:23:19.679185 11037 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:23:19.682258 11037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Bootstrap starting.
I20260807 04:23:19.682504 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 10966
I20260807 04:23:19.683302 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:34291
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:35625
--webserver_port=34907
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:23:19.700901 11037 log.cc:824] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:19.709293 11037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:23:19.709870 11037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Bootstrap complete.
I20260807 04:23:19.718686 11037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:19.719506 11037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ce7382576e434687e013700ed8137a, State: Initialized, Role: FOLLOWER
I20260807 04:23:19.720199 11037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:19.720434 11037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:23:19.720573 11037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:23:19.720773 11037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:23:19.724421 11037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:19.724903 11037 leader_election.cc:302] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ce7382576e434687e013700ed8137a; no voters:
I20260807 04:23:19.725596 11037 leader_election.cc:288] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:23:19.725739 11041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:23:19.726904 11041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [term 2 LEADER]: Becoming Leader. State: Replica: 79ce7382576e434687e013700ed8137a, State: Running, Role: LEADER
I20260807 04:23:19.727512 11041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } }
I20260807 04:23:19.727779 11037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:19.730398 11043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ce7382576e434687e013700ed8137a. Latest consensus state: current_term: 2 leader_uuid: "79ce7382576e434687e013700ed8137a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } } }
I20260807 04:23:19.730476 11042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "79ce7382576e434687e013700ed8137a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ce7382576e434687e013700ed8137a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35625 } } }
I20260807 04:23:19.730780 11043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:19.730822 11042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:19.742710 11047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:19.749284 11047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:19.750689 11047 catalog_manager.cc:1259] Loaded cluster ID: 5db6a1058daa43aba78e3ad967f432e2
I20260807 04:23:19.750846 11047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:19.754822 11047 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:19.756273 11047 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79ce7382576e434687e013700ed8137a: Loaded TSK: 0
I20260807 04:23:19.757839 11047 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:23:20.027529 11039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:20.027846 11039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:20.028043 11039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:20.035655 11039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:20.035812 11039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:20.036003 11039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:20.046463 11039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.193.keytab
--principal=oryx/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:34291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=34907
--tserver_master_addrs=127.1.244.254:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:20.047955 11039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:20.049297 11039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:20.063968 11065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:20.064249 11068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:20.063936 11066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:20.065313 11039 server_base.cc:1034] running on GCE node
I20260807 04:23:20.066011 11039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:20.066979 11039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:20.068207 11039 hybrid_clock.cc:648] HybridClock initialized: now 1786076600068139 us; error 56 us; skew 500 ppm
I20260807 04:23:20.068531 11039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:20 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076600, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:20.071820 11039 init.cc:377] Logged in from keytab as oryx/127.1.244.193@KRBTEST.COM (short username oryx)
I20260807 04:23:20.073467 11039 webserver.cc:460] Webserver started at http://127.1.244.193:34907/ using document root <none> and password file <none>
I20260807 04:23:20.074168 11039 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:20.074329 11039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:20.079818 11039 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:23:20.082531 11075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:20.083977 11039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:23:20.084161 11039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "41d5f83095bd43499b325472544494fd"
format_stamp: "Formatted at 2026-08-07 04:23:17 on dist-test-slave-zkpd"
server_key: "ffaaa74d9eebb282b09bdb34b947063b"
server_key_iv: "1823142eebb9ffe8a130512aeacf1d53"
server_key_version: "encryptionkey@0"
I20260807 04:23:20.085129 11039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:20.110996 11039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:20.114811 11039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:20.115105 11039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:20.116165 11039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:20.117491 11039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:20.117604 11039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:20.117744 11039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:20.117842 11039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:20.151439 11039 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34291
I20260807 04:23:20.151516 11190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34291 every 8 connection(s)
I20260807 04:23:20.152936 11039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:20.159122 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11039
I20260807 04:23:20.159734 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:45083
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:35625
--webserver_port=34533
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:23:20 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:20 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:23:20.496203 11195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:20.496459 11195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:20.496616 11195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:20.503620 11195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:20.503718 11195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:20.503981 11195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:20.513358 11195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.194.keytab
--principal=oryx/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:45083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=34533
--tserver_master_addrs=127.1.244.254:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:20.514637 11195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:20.516021 11195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:20.534770 11203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:20.534749 11202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:20.535064 11205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:20.536677 11195 server_base.cc:1034] running on GCE node
I20260807 04:23:20.537767 11195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:20.539377 11195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:20.540728 11195 hybrid_clock.cc:648] HybridClock initialized: now 1786076600540629 us; error 51 us; skew 500 ppm
I20260807 04:23:20.541270 11195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:20 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076600, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:20.546677 11195 init.cc:377] Logged in from keytab as oryx/127.1.244.194@KRBTEST.COM (short username oryx)
I20260807 04:23:20.549001 11195 webserver.cc:460] Webserver started at http://127.1.244.194:34533/ using document root <none> and password file <none>
I20260807 04:23:20.550261 11195 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:20.550472 11195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:20.560317 11195 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 04:23:20.564617 11212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:20.566669 11195 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:23:20.566946 11195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "c8ea744ba9b14ea39a713d5e52111671"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "b7c6961b0431319ff6e2ba92de739ba3"
server_key_iv: "f815b5dcaf0ff3cfb746b5354d65a671"
server_key_version: "encryptionkey@0"
I20260807 04:23:20.568987 11195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:20.598765 11195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:20.606237 11195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:20.606655 11195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:20.608317 11195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:20.610638 11195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:20.610983 11195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:20.611187 11195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:20.611323 11195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:20.651352 11328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45083 every 8 connection(s)
I20260807 04:23:20.651372 11195 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45083
I20260807 04:23:20.654006 11195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:20.655653 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11195
I20260807 04:23:20.656456 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:40459
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:35625
--webserver_port=40251
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:23:20 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:20 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:23:21.313941 11333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:21.314544 11333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:21.314776 11333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:21.333576 11333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:21.333813 11333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:21.334122 11333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:21.353677 11333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/krb5kdc/oryx_127.1.244.195.keytab
--principal=oryx/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:40459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=40251
--tserver_master_addrs=127.1.244.254:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:21.355351 11333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:21.357031 11333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:21.366492 11194 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.200s user 0.001s sys 0.012s
I20260807 04:23:21.368888 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:20.155942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:57217)
0807 04:23:20.156791 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:20.156845 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:20.158632 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:20.160094 (+ 1462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:20.160107 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:20.160711 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:20.161225 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:20.161242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.162844 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.162855 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:20.164115 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:20.164128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.165152 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.165161 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:20.165438 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:20.166116 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:20.166138 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:21.367575 (+1201437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":335,"threads_started":1}
W20260807 04:23:21.371515 11193 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:20.155924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:57217 (local address 127.1.244.254:35625)
0807 04:23:20.156727 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:20.156745 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:20.156970 (+ 225us) server_negotiation.cc:407] Connection header received
0807 04:23:20.159179 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:20.159192 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:20.159565 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:20.159964 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:20.161432 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.162667 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.164264 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.164916 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.165131 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.370505 (+1205374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:57217: BlockingRecv error: recv got EOF from 127.1.244.193:57217 (error 108)
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":301,"threads_started":1}
W20260807 04:23:21.372443 11191 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:35625 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.244.254:35625: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
W20260807 04:23:21.376803 11339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:21.378340 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:23:21.386641 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:21.387535 11333 server_base.cc:1034] running on GCE node
I20260807 04:23:21.389142 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:21.374075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:34021)
0807 04:23:21.375237 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:21.375262 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:21.375429 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:21.376844 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:21.376850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:21.377654 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:21.377985 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.378002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.379796 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.379804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.381904 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.381923 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.383481 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.383490 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.383776 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.384616 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:21.384634 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:21.388820 (+ 4186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1011,"thread_start_us":2941,"threads_started":1}
I20260807 04:23:21.389420 11333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260807 04:23:21.390650 11193 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:21.374520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:34021 (local address 127.1.244.254:35625)
0807 04:23:21.374738 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:21.374747 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:21.375547 (+ 800us) server_negotiation.cc:407] Connection header received
0807 04:23:21.375620 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:21.375628 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:21.375715 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:21.375829 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:21.378174 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.379534 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.382115 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.382998 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.383119 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.390410 (+ 7291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:34021: BlockingRecv error: recv got EOF from 127.1.244.193:34021 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:23:21.391530 11333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:21.392988 11333 hybrid_clock.cc:648] HybridClock initialized: now 1786076601392942 us; error 35 us; skew 500 ppm
I20260807 04:23:21.393330 11333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076601, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:21.397964 11333 init.cc:377] Logged in from keytab as oryx/127.1.244.195@KRBTEST.COM (short username oryx)
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:21.400851 11333 webserver.cc:460] Webserver started at http://127.1.244.195:40251/ using document root <none> and password file <none>
I20260807 04:23:21.401773 11333 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:21.401930 11333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:21.402825 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:21.390582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:43585)
0807 04:23:21.391063 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:21.391081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:21.391178 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:21.391657 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:21.391662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:21.392071 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:21.392273 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.392285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.394340 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.394347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.395332 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.395345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.396664 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.396672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.396868 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.397591 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:21.397614 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:21.402415 (+ 4801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368}
W20260807 04:23:21.403220 11191 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:23:21.403843 11193 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:21.391004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:43585 (local address 127.1.244.254:35625)
0807 04:23:21.391154 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:21.391159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:21.391175 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:23:21.391281 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:21.391288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:21.391386 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:21.391492 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:21.392459 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.394153 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.395487 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.396468 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.396646 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.403505 (+ 6859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:43585: BlockingRecv error: recv got EOF from 127.1.244.193:43585 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 04:23:21.408285 11333 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:23:21.412580 11351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:21.414688 11333 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:23:21.414953 11333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "23fb22f5bd8f4149a5a1a634dc43a482"
format_stamp: "Formatted at 2026-08-07 04:23:18 on dist-test-slave-zkpd"
server_key: "ef4f3191ceca11a713de536521f4342d"
server_key_iv: "4f2cea9a9899a8e0f41dd7d710fc44b9"
server_key_version: "encryptionkey@0"
I20260807 04:23:21.416945 11333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:21.442746 11333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:21.450127 11333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:21.450531 11333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:21.452147 11333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:21.454444 11333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:21.454605 11333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:21.454779 11333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:21.454900 11333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:21.500283 11468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40459 every 8 connection(s)
I20260807 04:23:21.500289 11333 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40459
I20260807 04:23:21.503005 11333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:21.504524 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11333
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:21.552996 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:21.505573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.1.244.254:35625)
0807 04:23:21.505796 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:21.505802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:21.532816 (+ 27014us) server_negotiation.cc:407] Connection header received
0807 04:23:21.533145 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:21.533169 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:21.533253 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:21.533367 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:21.536328 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.537181 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.538540 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.538950 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.539086 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.542429 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:21.542445 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:21.542475 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:21.542549 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:21.548208 (+ 5659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:21.549157 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:21.549171 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:21.549184 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:21.549486 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:21.549986 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:21.549994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:21.550000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:21.550864 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:21.551338 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:21.551844 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 04:23:21.552413 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 04:23:21.938704 11332 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.249s user 0.002s sys 0.024s
I20260807 04:23:21.941071 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:20.659029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:52061)
0807 04:23:20.673008 (+ 13979us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:20.673089 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:20.676898 (+ 3809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:20.678446 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:20.678465 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:20.679541 (+ 1076us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:20.680663 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:20.680695 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.683129 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.683138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:20.686184 (+ 3046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:20.686233 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.688212 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.688223 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:20.688665 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:20.689685 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:20.689736 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:21.939707 (+1249971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13511,"thread_start_us":14802,"threads_started":1}
W20260807 04:23:21.941390 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:20.658131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:52061 (local address 127.1.244.254:35625)
0807 04:23:20.673277 (+ 15146us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:20.673283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:20.673310 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:23:20.677491 (+ 4181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:20.677501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:20.677629 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:20.677770 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:20.681000 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.682853 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.686459 (+ 3606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:20.687485 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:20.687628 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.940427 (+1252799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:52061: BlockingRecv error: recv got EOF from 127.1.244.194:52061 (error 108)
Metrics: {"server-negotiator.queue_time_us":15000,"thread_start_us":15166,"threads_started":1}
W20260807 04:23:21.944567 11329 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:35625 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.244.254:35625: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:21.956420 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:21.945456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:42349)
0807 04:23:21.945957 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:21.945978 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:21.946094 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:21.946954 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:21.946962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:21.947648 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:21.948002 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.948023 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.949264 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.949270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.950426 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.950440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.951160 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.951166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.951354 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.952154 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:21.952172 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:21.956110 (+ 3938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260807 04:23:21.956532 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:21.945871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42349 (local address 127.1.244.254:35625)
0807 04:23:21.946310 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:21.946320 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:21.946346 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:23:21.946425 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:21.946433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:21.946543 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:21.946681 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:21.948181 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.949111 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.950571 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.950976 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.951107 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.956309 (+ 5202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:42349: BlockingRecv error: recv got EOF from 127.1.244.194:42349 (error 108)
Metrics: {"server-negotiator.queue_time_us":264}
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:21 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:23:21.969573 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:21.957315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41051 (local address 127.1.244.254:35625)
0807 04:23:21.957482 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:21.957486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:21.957500 (+ 14us) server_negotiation.cc:407] Connection header received
0807 04:23:21.957676 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:21.957679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:21.957737 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:21.957822 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:21.958974 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.959697 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.962356 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.962734 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.962833 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.969123 (+ 6290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:41051: BlockingRecv error: recv got EOF from 127.1.244.194:41051 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 04:23:21.970283 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:21.957131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:41051)
0807 04:23:21.957371 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:21.957399 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:21.957551 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:21.957919 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:21.957926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:21.958521 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:21.958801 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.958817 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.959852 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.959860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.962183 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.962206 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.962924 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.962934 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.963232 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.964045 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:21.964079 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:21.969875 (+ 5796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 04:23:21.970623 11329 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:22.414095 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:22.404192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:40531)
0807 04:23:22.404735 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:22.404750 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:22.404906 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:22.405385 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:22.405399 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:22.405963 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:22.406191 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.406203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.407395 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.407401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.408454 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.408466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.409200 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.409207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.409401 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.410212 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:22.410230 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:22.413759 (+ 3529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":334,"threads_started":1}
W20260807 04:23:22.414188 11331 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:22.404153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:40531 (local address 127.1.244.254:35625)
0807 04:23:22.404325 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:22.404329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:22.404859 (+ 530us) server_negotiation.cc:407] Connection header received
0807 04:23:22.405058 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:22.405066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:22.405204 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:22.405362 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:22.406339 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.407248 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.408593 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.409013 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.409165 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.413963 (+ 4798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:40531: BlockingRecv error: recv got EOF from 127.1.244.193:40531 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 04:23:22.645399 11472 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.103s user 0.002s sys 0.013s
W20260807 04:23:22.646615 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:21.508323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:42115 (local address 127.1.244.254:35625)
0807 04:23:21.529558 (+ 21235us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:21.529567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:21.530577 (+ 1010us) server_negotiation.cc:407] Connection header received
0807 04:23:21.533773 (+ 3196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:21.533778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:21.533920 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:21.534048 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:21.535602 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.536671 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.540185 (+ 3514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.540856 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.540997 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.646329 (+1105332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:42115: BlockingRecv error: recv got EOF from 127.1.244.195:42115 (error 108)
Metrics: {"server-negotiator.queue_time_us":21051,"thread_start_us":21253,"threads_started":1}
I20260807 04:23:22.647125 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:21.507564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:42115)
0807 04:23:21.530333 (+ 22769us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:21.530403 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:21.533461 (+ 3058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:21.534332 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:21.534341 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:21.534840 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:21.535428 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.535446 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.536868 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.536878 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.539995 (+ 3117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:21.540020 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:21.541179 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:21.541190 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:21.541629 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:21.542553 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:21.542580 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:22.646148 (+1103568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22376,"thread_start_us":22966,"threads_started":1}
W20260807 04:23:22.648201 11469 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:35625 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.244.254:35625: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:22.662184 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:22.648963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:39205)
0807 04:23:22.649139 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:22.649155 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:22.649248 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:22.649506 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:22.649509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:22.649845 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:22.650004 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.650012 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.651023 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.651028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.652094 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.652108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.652838 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.652844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.653013 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.658249 (+ 5236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:22.658266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:22.661997 (+ 3731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 04:23:22.662411 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:22.649027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:39205 (local address 127.1.244.254:35625)
0807 04:23:22.649264 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:22.649268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:22.649285 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:23:22.649330 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:22.649334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:22.649391 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:22.649467 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:22.650123 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.650848 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.652238 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.652654 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.652784 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.662172 (+ 9388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:39205: BlockingRecv error: recv got EOF from 127.1.244.195:39205 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:22.670265 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:22.662871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:58451)
0807 04:23:22.663038 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:22.663053 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:22.663128 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:22.663369 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:22.663372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:22.663639 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:22.663796 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.663803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.664861 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.664866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.665828 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.665839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.666498 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.666502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.666615 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.667070 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:22.667085 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:22.669999 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 04:23:22.670465 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:22.662933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58451 (local address 127.1.244.254:35625)
0807 04:23:22.663078 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:22.663082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:22.663097 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:23:22.663217 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:22.663220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:22.663272 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:22.663353 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:22.663945 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.664726 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.665944 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.666377 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.666461 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.670182 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:58451: BlockingRecv error: recv got EOF from 127.1.244.195:58451 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 04:23:22.670719 11469 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:22 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:22.980628 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:22.971190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:43481)
0807 04:23:22.972039 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:22.972056 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:22.972221 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:22.972599 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:22.972604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:22.973065 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:22.973206 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.973215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.974265 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.974271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.975238 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:22.975250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.976037 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.976044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:22.976220 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.976842 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:22.976860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:22.980395 (+ 3535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":582,"threads_started":1}
W20260807 04:23:22.980775 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:22.971285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43481 (local address 127.1.244.254:35625)
0807 04:23:22.971543 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:22.971553 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:22.972149 (+ 596us) server_negotiation.cc:407] Connection header received
0807 04:23:22.972363 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:22.972375 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:22.972457 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:22.972563 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:22.973354 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.974108 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.975389 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:22.975823 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:22.975988 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:22.980580 (+ 4592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:43481: BlockingRecv error: recv got EOF from 127.1.244.194:43481 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 04:23:23 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:23 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:23.424094 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:23.415114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:45039)
0807 04:23:23.415584 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:23.415601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:23.415726 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:23.416114 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:23.416118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:23.416667 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:23.416958 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:23.416992 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.418150 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.418155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:23.419069 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:23.419079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.419747 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.419751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:23.419935 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:23.420504 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:23.420521 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:23.423783 (+ 3262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":299,"threads_started":1}
W20260807 04:23:23.424307 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:23.415193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:45039 (local address 127.1.244.254:35625)
0807 04:23:23.415441 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:23.415448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:23.415689 (+ 241us) server_negotiation.cc:407] Connection header received
0807 04:23:23.415828 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:23.415835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:23.415965 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:23.416074 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:23.417124 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.417992 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.419181 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.419614 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.419709 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:23.424068 (+ 4359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:45039: BlockingRecv error: recv got EOF from 127.1.244.193:45039 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 04:23:23 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:23 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:23.681692 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:23.671347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:38051)
0807 04:23:23.671965 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:23.671985 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:23.672113 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:23.672410 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:23.672415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:23.672933 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:23.673120 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:23.673130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.674122 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.674129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:23.675665 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:23.675680 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.676476 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.676487 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:23.676668 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:23.677425 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:23.677447 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:23.681462 (+ 4015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":331,"threads_started":1}
W20260807 04:23:23.681903 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:23.671435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:38051 (local address 127.1.244.254:35625)
0807 04:23:23.671678 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:23.671687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:23.672054 (+ 367us) server_negotiation.cc:407] Connection header received
0807 04:23:23.672216 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:23.672219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:23.672281 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:23.672360 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:23.673248 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.674001 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.675799 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.676287 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.676381 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:23.681672 (+ 5291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:38051: BlockingRecv error: recv got EOF from 127.1.244.195:38051 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 04:23:23 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:23 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:23.990849 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:23.981622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:34979)
0807 04:23:23.982097 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:23.982112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:23.982231 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:23.982542 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:23.982546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:23.982935 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:23.983124 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:23.983132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.984183 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.984187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:23.985100 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:23.985112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.985803 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.985807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:23.985931 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:23.986596 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:23.986621 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:23.990594 (+ 3973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":284,"threads_started":1}
W20260807 04:23:23.991027 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:23.981683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:34979 (local address 127.1.244.254:35625)
0807 04:23:23.981916 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:23.981926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:23.982198 (+ 272us) server_negotiation.cc:407] Connection header received
0807 04:23:23.982339 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:23.982343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:23.982412 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:23.982497 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:23.983275 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.984071 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.985251 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:23.985670 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:23.985772 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:23.990787 (+ 5015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:34979: BlockingRecv error: recv got EOF from 127.1.244.194:34979 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 04:23:24 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:24 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:24.433957 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:24.425161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:50143)
0807 04:23:24.425710 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:24.425726 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:24.425839 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:24.426123 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:24.426126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:24.426487 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:24.426661 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:24.426669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.427600 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.427603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:24.428685 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:24.428698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.429414 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.429419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:24.429551 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:24.430197 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:24.430215 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:24.433553 (+ 3338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":305,"threads_started":1}
W20260807 04:23:24.434067 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:24.425408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:50143 (local address 127.1.244.254:35625)
0807 04:23:24.425589 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:24.425593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:24.425794 (+ 201us) server_negotiation.cc:407] Connection header received
0807 04:23:24.425934 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:24.425938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:24.425996 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:24.426077 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:24.426779 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.427491 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.428846 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.429263 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.429351 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:24.433808 (+ 4457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:50143: BlockingRecv error: recv got EOF from 127.1.244.193:50143 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 04:23:24 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:24 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:24.691876 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:24.682718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:59173)
0807 04:23:24.683252 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:24.683268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:24.683385 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:24.683775 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:24.683782 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:24.684417 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:24.684664 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:24.684681 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.685867 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.685871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:24.686775 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:24.686786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.687432 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.687436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:24.687561 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:24.688138 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:24.688156 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:24.691635 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":311,"threads_started":1}
W20260807 04:23:24.692098 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:24.682778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59173 (local address 127.1.244.254:35625)
0807 04:23:24.682940 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:24.682945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:24.683350 (+ 405us) server_negotiation.cc:407] Connection header received
0807 04:23:24.683503 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:24.683508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:24.683587 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:24.683702 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:24.684819 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.685750 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.686899 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.687300 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.687390 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:24.691798 (+ 4408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:59173: BlockingRecv error: recv got EOF from 127.1.244.195:59173 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 04:23:24 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:24 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:25.000059 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:24.991794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:36357)
0807 04:23:24.992353 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:24.992368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:24.992484 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:24.992781 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:24.992785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:24.993140 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:24.993308 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:24.993317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.994273 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.994276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:24.995170 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:24.995181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.995786 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.995790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:24.995951 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:24.996492 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:24.996510 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:24.999672 (+ 3162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":298,"threads_started":1}
W20260807 04:23:25.000164 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:24.991850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:36357 (local address 127.1.244.254:35625)
0807 04:23:24.992082 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:24.992091 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:24.992449 (+ 358us) server_negotiation.cc:407] Connection header received
0807 04:23:24.992585 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:24.992588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:24.992658 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:24.992739 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:24.993431 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.994167 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.995286 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:24.995671 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:24.995757 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:24.999870 (+ 4113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:36357: BlockingRecv error: recv got EOF from 127.1.244.194:36357 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 04:23:25 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:25 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:25.444100 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:25.435126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:51707)
0807 04:23:25.435653 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:25.435668 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:25.435837 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:25.436235 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:25.436239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:25.436657 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:25.436913 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:25.436930 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.437943 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.437949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:25.438950 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:25.438962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.439667 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.439674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:25.439874 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:25.440474 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:25.440492 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:25.443722 (+ 3230us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":290,"threads_started":1}
W20260807 04:23:25.444221 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:25.435200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51707 (local address 127.1.244.254:35625)
0807 04:23:25.435441 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:25.435450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:25.435760 (+ 310us) server_negotiation.cc:407] Connection header received
0807 04:23:25.436014 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:25.436021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:25.436088 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:25.436203 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:25.437064 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.437798 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.439114 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.439516 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.439658 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:25.443988 (+ 4330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:51707: BlockingRecv error: recv got EOF from 127.1.244.193:51707 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 04:23:25 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:25 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:25.701857 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:25.693087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:37731)
0807 04:23:25.693590 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:25.693607 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:25.693740 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:25.694044 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:25.694046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:25.694387 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:25.694557 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:25.694564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.695545 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.695548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:25.696524 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:25.696536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.697232 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.697238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:25.697445 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:25.698129 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:25.698152 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:25.701618 (+ 3466us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":286,"threads_started":1}
W20260807 04:23:25.702013 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:25.693153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:37731 (local address 127.1.244.254:35625)
0807 04:23:25.693388 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:25.693394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:25.693702 (+ 308us) server_negotiation.cc:407] Connection header received
0807 04:23:25.693846 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:25.693850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:25.693919 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:25.694001 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:25.694679 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.695426 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.696654 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:25.697072 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:25.697213 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:25.701806 (+ 4593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:37731: BlockingRecv error: recv got EOF from 127.1.244.195:37731 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 04:23:26 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:26 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:26.010700 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:26.001128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:44971)
0807 04:23:26.001689 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:26.001704 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:26.001881 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:26.002233 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:26.002237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:26.002645 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:26.002808 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:26.002817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.003856 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.003863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:26.005215 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:26.005231 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.006195 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.006201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:26.006344 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:26.006910 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:26.006927 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:26.010447 (+ 3520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":307,"threads_started":1}
W20260807 04:23:26.010998 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:26.001242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44971 (local address 127.1.244.254:35625)
0807 04:23:26.001474 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:26.001482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:26.001816 (+ 334us) server_negotiation.cc:407] Connection header received
0807 04:23:26.002014 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:26.002019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:26.002088 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:26.002191 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:26.002963 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.003713 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.005365 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.006018 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.006155 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:26.010684 (+ 4529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:44971: BlockingRecv error: recv got EOF from 127.1.244.194:44971 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 04:23:26 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:26 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:26.454128 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:26.445047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:39001)
0807 04:23:26.445563 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:26.445579 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:26.445749 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:26.446088 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:26.446111 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:26.446555 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:26.446711 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:26.446720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.447740 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.447747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:26.448738 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:26.448750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.449620 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.449637 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:26.449843 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:26.450442 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:26.450459 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:26.453890 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":261,"threads_started":1}
W20260807 04:23:26.454278 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:26.445105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:39001 (local address 127.1.244.254:35625)
0807 04:23:26.445330 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:26.445339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:26.445667 (+ 328us) server_negotiation.cc:407] Connection header received
0807 04:23:26.445871 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:26.445876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:26.445952 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:26.446052 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:26.446857 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.447598 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.448901 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.449467 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.449598 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:26.454075 (+ 4477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:39001: BlockingRecv error: recv got EOF from 127.1.244.193:39001 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 04:23:26 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:26 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:26.711647 11488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:26.702874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:40915)
0807 04:23:26.703381 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:26.703398 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:26.703518 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:26.703930 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:26.703936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:26.704340 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:26.704533 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:26.704542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.705793 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.705797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:26.706777 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:26.706789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.707422 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.707427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:26.707547 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:26.708170 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:26.708187 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:26.711400 (+ 3213us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":292,"threads_started":1}
W20260807 04:23:26.711880 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:26.703121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:40915 (local address 127.1.244.254:35625)
0807 04:23:26.703342 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:26.703349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:26.703484 (+ 135us) server_negotiation.cc:407] Connection header received
0807 04:23:26.703646 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:26.703649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:26.703724 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:26.703803 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:26.704698 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.705671 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.706901 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:26.707301 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:26.707389 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:26.711598 (+ 4209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:40915: BlockingRecv error: recv got EOF from 127.1.244.195:40915 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 04:23:27 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:27 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:27.020661 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:27.011675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:47941)
0807 04:23:27.012316 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:27.012338 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:27.012451 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:27.012846 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:27.012850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:27.013511 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:27.013764 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:27.013782 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.014740 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.014744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:27.015654 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:27.015666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.016301 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.016305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:27.016435 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:27.017057 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:27.017075 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:27.020442 (+ 3367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":367,"threads_started":1}
W20260807 04:23:27.020843 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:27.011935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:47941 (local address 127.1.244.254:35625)
0807 04:23:27.012164 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:27.012170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:27.012420 (+ 250us) server_negotiation.cc:407] Connection header received
0807 04:23:27.012590 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:27.012596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:27.012664 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:27.012780 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:27.013915 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.014626 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.015775 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.016180 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.016262 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:27.020630 (+ 4368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:47941: BlockingRecv error: recv got EOF from 127.1.244.194:47941 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 04:23:27 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:27 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:27.466305 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:27.455113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:47363)
0807 04:23:27.455845 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:27.455862 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:27.456069 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:27.456449 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:27.456456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:27.456893 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:27.457063 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:27.457073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.458120 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.458128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:27.460001 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:27.460017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.460785 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.460791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:27.460982 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:27.461705 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:27.461730 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:27.465961 (+ 4231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":495,"threads_started":1}
W20260807 04:23:27.466419 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:27.455196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:47363 (local address 127.1.244.254:35625)
0807 04:23:27.455458 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:27.455467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:27.455984 (+ 517us) server_negotiation.cc:407] Connection header received
0807 04:23:27.456217 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:27.456225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:27.456293 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:27.456400 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:27.457220 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.457972 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.460171 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.460622 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.460768 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:27.466166 (+ 5398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:47363: BlockingRecv error: recv got EOF from 127.1.244.193:47363 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 04:23:27 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:27 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:27.725787 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:27.712603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:45903)
0807 04:23:27.713077 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:27.713093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:27.713205 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:27.713483 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:27.713487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:27.713948 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:27.714093 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:27.714102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.715071 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.715075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:27.716073 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:27.716085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.716703 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.716708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:27.716836 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:27.721397 (+ 4561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:27.721435 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:27.725461 (+ 4026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":275,"threads_started":1}
W20260807 04:23:27.725898 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:27.712707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:45903 (local address 127.1.244.254:35625)
0807 04:23:27.712949 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:27.712955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:27.713170 (+ 215us) server_negotiation.cc:407] Connection header received
0807 04:23:27.713288 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:27.713291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:27.713353 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:27.713436 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:27.714213 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.714954 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.716190 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:27.716587 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:27.716679 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:27.725665 (+ 8986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:45903: BlockingRecv error: recv got EOF from 127.1.244.195:45903 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 07 04:23:28 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:28 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:28.030503 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:28.021598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:35325)
0807 04:23:28.022081 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:28.022096 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:28.022257 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:28.022610 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:28.022615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:28.023048 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:28.023217 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:28.023226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.024288 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.024298 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:28.025294 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:28.025306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.026018 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.026025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:28.026209 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:28.026800 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:28.026817 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:28.030144 (+ 3327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":274,"threads_started":1}
W20260807 04:23:28.030639 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:28.021635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35325 (local address 127.1.244.254:35625)
0807 04:23:28.021855 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:28.021863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:28.022182 (+ 319us) server_negotiation.cc:407] Connection header received
0807 04:23:28.022394 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:28.022399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:28.022469 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:28.022574 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:28.023359 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.024142 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.025452 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.025846 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.025983 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:28.030379 (+ 4396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:35325: BlockingRecv error: recv got EOF from 127.1.244.194:35325 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 04:23:28 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:28 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 04:23:28.477339 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:28.467443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:55623 (local address 127.1.244.254:35625)
0807 04:23:28.467623 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:28.467627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:28.468168 (+ 541us) server_negotiation.cc:407] Connection header received
0807 04:23:28.468378 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:28.468382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:28.468449 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:28.468538 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:28.469341 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.470057 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.471276 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.471662 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.471751 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:28.477081 (+ 5330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:55623: BlockingRecv error: recv got EOF from 127.1.244.193:55623 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:23:28.477339 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:28.467406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:55623)
0807 04:23:28.468073 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:28.468090 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:28.468226 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:28.468583 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:28.468587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:28.469017 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:28.469198 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:28.469207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.470198 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.470201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:28.471154 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:28.471165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.471958 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.471969 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:28.472118 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:28.472786 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:28.472804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:28.476874 (+ 4070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":340,"threads_started":1}
Aug 07 04:23:28 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:28 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:28.736035 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:28.726917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:56645)
0807 04:23:28.727404 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:28.727437 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:28.727594 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:28.727982 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:28.727989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:28.728426 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:28.728585 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:28.728593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.729673 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.729680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:28.730606 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:28.730617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.731360 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.731367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:28.731551 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:28.732127 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:28.732144 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:28.735728 (+ 3584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":267,"threads_started":1}
W20260807 04:23:28.736225 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:28.727010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56645 (local address 127.1.244.254:35625)
0807 04:23:28.727232 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:28.727240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:28.727521 (+ 281us) server_negotiation.cc:407] Connection header received
0807 04:23:28.727721 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:28.727727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:28.727801 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:28.727933 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:28.728740 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.729524 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.730750 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:28.731199 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:28.731324 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:28.735973 (+ 4649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:56645: BlockingRecv error: recv got EOF from 127.1.244.195:56645 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 04:23:29 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:29 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:29.040839 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:29.031522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:58043)
0807 04:23:29.032257 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:29.032278 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:29.032457 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:29.032807 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:29.032812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:29.033292 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:29.033519 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:29.033529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.034630 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.034636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:29.035594 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:29.035606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.036410 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.036418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:29.036596 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:29.037237 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:29.037255 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:29.040592 (+ 3337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":348,"threads_started":1}
W20260807 04:23:29.040968 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:29.031615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:58043 (local address 127.1.244.254:35625)
0807 04:23:29.031843 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:29.031849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:29.032366 (+ 517us) server_negotiation.cc:407] Connection header received
0807 04:23:29.032591 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:29.032599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:29.032664 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:29.032773 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:29.033660 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.034486 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.035745 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.036243 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.036388 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:29.040779 (+ 4391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:58043: BlockingRecv error: recv got EOF from 127.1.244.194:58043 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 04:23:29 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:29 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:29.487771 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:29.478764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:44935)
0807 04:23:29.479454 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:29.479470 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:29.479633 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:29.479991 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:29.479997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:29.480418 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:29.480582 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:29.480592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.481608 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.481615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:29.482542 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:29.482554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.483300 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.483306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:29.483464 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:29.484118 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:29.484136 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:29.487379 (+ 3243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":419,"threads_started":1}
W20260807 04:23:29.487865 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:29.478788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:44935 (local address 127.1.244.254:35625)
0807 04:23:29.479014 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:29.479023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:29.479566 (+ 543us) server_negotiation.cc:407] Connection header received
0807 04:23:29.479756 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:29.479761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:29.479832 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:29.479959 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:29.480727 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.481451 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.482682 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.483126 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.483265 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:29.487589 (+ 4324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:44935: BlockingRecv error: recv got EOF from 127.1.244.193:44935 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 04:23:29 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:29 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:29.745923 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:29.737122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:52371)
0807 04:23:29.737749 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:29.737773 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:29.737919 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:29.738301 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:29.738304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:29.738871 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:29.739049 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:29.739058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.740047 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.740051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:29.740983 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:29.740994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.741623 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.741627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:29.741735 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:29.742272 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:29.742289 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:29.745681 (+ 3392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":272,"threads_started":1}
W20260807 04:23:29.746057 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:29.737306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52371 (local address 127.1.244.254:35625)
0807 04:23:29.737555 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:29.737562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:29.737866 (+ 304us) server_negotiation.cc:407] Connection header received
0807 04:23:29.738046 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:29.738053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:29.738139 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:29.738256 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:29.739187 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.739929 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.741097 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:29.741502 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:29.741590 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:29.745845 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:52371: BlockingRecv error: recv got EOF from 127.1.244.195:52371 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 04:23:30 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:30 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:30.050825 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:30.041770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:54523)
0807 04:23:30.042291 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:30.042305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:30.042483 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:30.042820 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:30.042825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:30.043378 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:30.043559 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:30.043568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.044926 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.044932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:30.045820 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:30.045831 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.046563 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.046569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:30.046740 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:30.047294 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:30.047311 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:30.050579 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":269,"threads_started":1}
W20260807 04:23:30.050997 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:30.041838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:54523 (local address 127.1.244.254:35625)
0807 04:23:30.042093 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:30.042101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:30.042400 (+ 299us) server_negotiation.cc:407] Connection header received
0807 04:23:30.042615 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:30.042620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:30.042690 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:30.042790 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:30.043704 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.044783 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.045968 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.046372 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.046521 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:30.050769 (+ 4248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:54523: BlockingRecv error: recv got EOF from 127.1.244.194:54523 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 04:23:30 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:30 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:30.497985 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:30.488881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:48279)
0807 04:23:30.489434 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:30.489449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:30.489628 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:30.490034 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:30.490039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:30.490479 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:30.490627 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:30.490636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.491643 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.491649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:30.492995 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:30.493014 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.493742 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.493746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:30.493872 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:30.494436 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:30.494454 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:30.497754 (+ 3300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":301,"threads_started":1}
W20260807 04:23:30.498181 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:30.488965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:48279 (local address 127.1.244.254:35625)
0807 04:23:30.489211 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:30.489219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:30.489552 (+ 333us) server_negotiation.cc:407] Connection header received
0807 04:23:30.489801 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:30.489805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:30.489867 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:30.489982 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:30.490768 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.491503 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.493143 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.493598 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.493708 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:30.497964 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:48279: BlockingRecv error: recv got EOF from 127.1.244.193:48279 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 04:23:30 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:30 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:30.755949 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:30.746976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:48161)
0807 04:23:30.747550 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:30.747566 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:30.747681 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:30.748025 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:30.748028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:30.748425 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:30.748590 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:30.748598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.749571 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.749574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:30.750629 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:30.750642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.751695 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.751702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:30.751839 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:30.752476 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:30.752494 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:30.755646 (+ 3152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":299,"threads_started":1}
W20260807 04:23:30.756122 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:30.747050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:48161 (local address 127.1.244.254:35625)
0807 04:23:30.747209 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:30.747214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:30.747633 (+ 419us) server_negotiation.cc:407] Connection header received
0807 04:23:30.747775 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:30.747778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:30.747837 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:30.747969 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:30.748706 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.749453 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.750798 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:30.751473 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:30.751600 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:30.755830 (+ 4230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:48161: BlockingRecv error: recv got EOF from 127.1.244.195:48161 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 04:23:31 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:31 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:31.061178 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:31.051844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:46621)
0807 04:23:31.052382 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:31.052398 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:31.052560 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:31.052929 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:31.052934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:31.053381 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:31.053535 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:31.053543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.054562 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.054566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:31.056022 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:31.056036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.056687 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.056691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:31.056807 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:31.057454 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:31.057472 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:31.060950 (+ 3478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":307,"threads_started":1}
W20260807 04:23:31.061355 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:31.051916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:46621 (local address 127.1.244.254:35625)
0807 04:23:31.052150 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:31.052158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:31.052505 (+ 347us) server_negotiation.cc:407] Connection header received
0807 04:23:31.052703 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:31.052709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:31.052779 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:31.052881 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:31.053695 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.054444 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.056152 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.056560 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.056656 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:31.061128 (+ 4472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:46621: BlockingRecv error: recv got EOF from 127.1.244.194:46621 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 04:23:31 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:31 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:31.508500 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:31.499020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:34159)
0807 04:23:31.499591 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:31.499612 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:31.499752 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:31.500185 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:31.500192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:31.500746 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:31.500986 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:31.501002 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.502161 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.502168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:31.503600 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:31.503614 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.504333 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.504338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:31.504450 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:31.505072 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:31.505089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:31.508293 (+ 3204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":242,"threads_started":1}
W20260807 04:23:31.508744 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:31.499092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:34159 (local address 127.1.244.254:35625)
0807 04:23:31.499316 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:31.499323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:31.499713 (+ 390us) server_negotiation.cc:407] Connection header received
0807 04:23:31.499883 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:31.499926 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:31.500005 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:31.500125 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:31.501246 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.502020 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.503726 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.504193 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.504292 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:31.508460 (+ 4168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:34159: BlockingRecv error: recv got EOF from 127.1.244.193:34159 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 04:23:31 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:31 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:31.765728 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:31.756852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:49901)
0807 04:23:31.757381 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:31.757398 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:31.757510 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:31.757809 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:31.757813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:31.758312 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:31.758474 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:31.758484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.759574 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.759580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:31.760615 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:31.760629 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.761373 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.761380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:31.761556 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:31.762101 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:31.762119 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:31.765393 (+ 3274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":287,"threads_started":1}
W20260807 04:23:31.765825 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:31.756984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:49901 (local address 127.1.244.254:35625)
0807 04:23:31.757171 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:31.757175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:31.757471 (+ 296us) server_negotiation.cc:407] Connection header received
0807 04:23:31.757605 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:31.757609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:31.757667 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:31.757747 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:31.758641 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.759416 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.760759 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:31.761206 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:31.761339 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:31.765605 (+ 4266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:49901: BlockingRecv error: recv got EOF from 127.1.244.195:49901 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 04:23:32 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:32 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:32.071367 11504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:32.062181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:37125)
0807 04:23:32.062743 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:32.062760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:32.062878 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:32.063173 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:32.063177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:32.063547 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:32.063714 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:32.063722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.064924 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.064930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:32.066319 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:32.066332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.067031 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.067035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:32.067137 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:32.067671 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:32.067688 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:32.071017 (+ 3329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":325,"threads_started":1}
W20260807 04:23:32.071476 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:32.062279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:37125 (local address 127.1.244.254:35625)
0807 04:23:32.062474 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:32.062478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:32.062834 (+ 356us) server_negotiation.cc:407] Connection header received
0807 04:23:32.062981 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:32.062984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:32.063042 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:32.063123 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:32.063831 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.064733 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.066446 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.066910 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.067000 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:32.071200 (+ 4200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:37125: BlockingRecv error: recv got EOF from 127.1.244.194:37125 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 04:23:32 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:32 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:32.518730 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:32.509648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:42971)
0807 04:23:32.510226 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:32.510240 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:32.510393 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:32.510750 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:32.510755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:32.511202 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:32.511377 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:32.511386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.512460 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.512468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:32.513540 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:32.513557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.514284 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.514289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:32.514482 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:32.515070 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:32.515087 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:32.518401 (+ 3314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":322,"threads_started":1}
W20260807 04:23:32.518849 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:32.509817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:42971 (local address 127.1.244.254:35625)
0807 04:23:32.510082 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:32.510089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:32.510322 (+ 233us) server_negotiation.cc:407] Connection header received
0807 04:23:32.510517 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:32.510522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:32.510590 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:32.510715 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:32.511522 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.512328 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.513700 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.514122 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.514255 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:32.518591 (+ 4336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:42971: BlockingRecv error: recv got EOF from 127.1.244.193:42971 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 07 04:23:32 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:32 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:32.780200 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:32.766940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:36849)
0807 04:23:32.767451 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:32.767466 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:32.767626 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:32.767985 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:32.767989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:32.768397 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:32.768580 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:32.768590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.769653 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.769660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:32.770658 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:32.770670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.771396 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.771402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:32.771577 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:32.776404 (+ 4827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:32.776429 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:32.779955 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":268,"threads_started":1}
W20260807 04:23:32.780409 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:32.767006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:36849 (local address 127.1.244.254:35625)
0807 04:23:32.767240 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:32.767248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:32.767558 (+ 310us) server_negotiation.cc:407] Connection header received
0807 04:23:32.767760 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:32.767765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:32.767824 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:32.767963 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:32.768730 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.769505 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.770808 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:32.771240 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:32.771373 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:32.780141 (+ 8768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:36849: BlockingRecv error: recv got EOF from 127.1.244.195:36849 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 04:23:33 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:33 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:33.081777 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:33.072464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:35363)
0807 04:23:33.072987 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:33.073007 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:33.073132 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:33.073428 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:33.073431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:33.073858 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:33.074048 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:33.074057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.075030 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.075034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:33.075965 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:33.075977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.077006 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.077013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:33.077188 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:33.077825 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:33.077886 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:33.081449 (+ 3563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":296,"threads_started":1}
W20260807 04:23:33.081845 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:33.072538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35363 (local address 127.1.244.254:35625)
0807 04:23:33.072735 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:33.072739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:33.073093 (+ 354us) server_negotiation.cc:407] Connection header received
0807 04:23:33.073235 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:33.073238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:33.073296 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:33.073377 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:33.074169 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.074875 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.076136 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.076818 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.076990 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:33.081615 (+ 4625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:35363: BlockingRecv error: recv got EOF from 127.1.244.194:35363 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 04:23:33 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:33 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:33.529011 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:33.519792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:59153)
0807 04:23:33.520347 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:33.520363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:33.520519 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:33.520886 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:33.520892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:33.521323 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:33.521498 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:33.521507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.522550 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.522556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:33.523531 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:33.523543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.524363 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.524369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:33.524588 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:33.525310 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:33.525328 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:33.528723 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":315,"threads_started":1}
W20260807 04:23:33.529227 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:33.519852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:59153 (local address 127.1.244.254:35625)
0807 04:23:33.520102 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:33.520110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:33.520448 (+ 338us) server_negotiation.cc:407] Connection header received
0807 04:23:33.520638 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:33.520643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:33.520711 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:33.520834 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:33.521644 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.522398 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.523723 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.524163 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.524312 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:33.528941 (+ 4629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:59153: BlockingRecv error: recv got EOF from 127.1.244.193:59153 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 04:23:33 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:33 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:33.790452 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:33.781084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:47887)
0807 04:23:33.781584 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:33.781599 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:33.781756 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:33.782101 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:33.782106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:33.782567 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:33.782749 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:33.782759 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.783989 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.783995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:33.785150 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:33.785162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.785889 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.785895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:33.786095 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:33.786655 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:33.786673 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:33.790125 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
W20260807 04:23:33.790573 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:33.781193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47887 (local address 127.1.244.254:35625)
0807 04:23:33.781423 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:33.781431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:33.781685 (+ 254us) server_negotiation.cc:407] Connection header received
0807 04:23:33.781885 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:33.781896 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:33.781957 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:33.782060 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:33.782905 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.783828 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.785293 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:33.785716 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:33.785874 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:33.790318 (+ 4444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:47887: BlockingRecv error: recv got EOF from 127.1.244.195:47887 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 04:23:34 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:34 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:34.092132 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:34.082702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:48203)
0807 04:23:34.083278 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:34.083294 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:34.083409 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:34.083722 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:34.083725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:34.084140 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:34.084306 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:34.084315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.085313 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.085316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:34.086244 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:34.086255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.087019 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.087023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:34.087151 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:34.087799 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:34.087827 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:34.091755 (+ 3928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":327,"threads_started":1}
W20260807 04:23:34.092252 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:34.082758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:48203 (local address 127.1.244.254:35625)
0807 04:23:34.082941 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:34.082946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:34.083370 (+ 424us) server_negotiation.cc:407] Connection header received
0807 04:23:34.083523 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:34.083526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:34.083586 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:34.083667 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:34.084434 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.085190 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.086398 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.086852 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.086964 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:34.091979 (+ 5015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:48203: BlockingRecv error: recv got EOF from 127.1.244.194:48203 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 04:23:34 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:34 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:34.539036 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:34.529898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:47467)
0807 04:23:34.530437 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:34.530454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:34.530610 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:34.530964 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:34.530969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:34.531416 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:34.531560 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:34.531569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.532668 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.532674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:34.533663 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:34.533675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.534462 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.534468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:34.534642 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:34.535198 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:34.535214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:34.538759 (+ 3545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":282,"threads_started":1}
W20260807 04:23:34.539168 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:34.529965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:47467 (local address 127.1.244.254:35625)
0807 04:23:34.530185 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:34.530192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:34.530542 (+ 350us) server_negotiation.cc:407] Connection header received
0807 04:23:34.530734 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:34.530740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:34.530817 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:34.530936 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:34.531704 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.532529 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.533834 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.534320 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.534447 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:34.538936 (+ 4489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:47467: BlockingRecv error: recv got EOF from 127.1.244.193:47467 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 04:23:34 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:34 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:34.800365 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:34.791421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:37149)
0807 04:23:34.792030 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:34.792047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:34.792158 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:34.792447 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:34.792450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:34.792898 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:34.793090 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:34.793099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.794096 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.794101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:34.795075 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:34.795088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.795731 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.795735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:34.795860 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:34.796496 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:34.796512 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:34.800056 (+ 3544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":331,"threads_started":1}
W20260807 04:23:34.800550 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:34.791483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:37149 (local address 127.1.244.254:35625)
0807 04:23:34.791650 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:34.791655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:34.792120 (+ 465us) server_negotiation.cc:407] Connection header received
0807 04:23:34.792251 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:34.792255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:34.792325 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:34.792403 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:34.793205 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.793964 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.795195 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:34.795583 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:34.795673 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:34.800289 (+ 4616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:37149: BlockingRecv error: recv got EOF from 127.1.244.195:37149 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 04:23:35 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:35 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:35.103737 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:35.093231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:39955)
0807 04:23:35.093765 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:35.093785 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:35.093935 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:35.094309 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:35.094313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:35.094722 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:35.094873 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:35.094882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.096403 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.096412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:35.097316 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:35.097326 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.098432 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.098436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:35.098558 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:35.099197 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:35.099214 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:35.103335 (+ 4121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":272,"threads_started":1}
W20260807 04:23:35.103789 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:35.093514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:39955 (local address 127.1.244.254:35625)
0807 04:23:35.093750 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:35.093756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:35.093846 (+ 90us) server_negotiation.cc:407] Connection header received
0807 04:23:35.094066 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:35.094074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:35.094163 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:35.094272 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:35.095045 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.096276 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.097468 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.098283 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.098420 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:35.103513 (+ 5093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:39955: BlockingRecv error: recv got EOF from 127.1.244.194:39955 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 04:23:35 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:35 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.193@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:35.549284 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:35.540497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.193:53459)
0807 04:23:35.541033 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:35.541050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:35.541160 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:35.541448 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:35.541451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:35.541807 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:35.542003 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:35.542012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.543007 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.543011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:35.544349 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:35.544383 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.545166 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.545170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:35.545297 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:35.545855 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:35.545889 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:35.549029 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":281,"threads_started":1}
W20260807 04:23:35.549423 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:35.540629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53459 (local address 127.1.244.254:35625)
0807 04:23:35.540845 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:35.540849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:35.541123 (+ 274us) server_negotiation.cc:407] Connection header received
0807 04:23:35.541253 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:35.541257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:35.541315 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:35.541395 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:35.542124 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.542878 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.544484 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.545041 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.545132 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:35.549210 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.193:53459: BlockingRecv error: recv got EOF from 127.1.244.193:53459 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 04:23:35 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:35 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.195@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:35.810084 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:35.801317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.195:35387)
0807 04:23:35.801820 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:35.801836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:35.802002 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:35.802383 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:35.802394 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:35.802856 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:35.803040 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:35.803053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.804117 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.804120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:35.805022 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:35.805034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.805645 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.805649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:35.805771 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:35.806329 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:35.806346 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:35.809822 (+ 3476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":267,"threads_started":1}
W20260807 04:23:35.810276 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:35.801412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35387 (local address 127.1.244.254:35625)
0807 04:23:35.801714 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:35.801721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:35.801919 (+ 198us) server_negotiation.cc:407] Connection header received
0807 04:23:35.802124 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:35.802129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:35.802203 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:35.802340 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:35.803188 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.803967 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.805139 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:35.805526 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:35.805617 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:35.810038 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.195:35387: BlockingRecv error: recv got EOF from 127.1.244.195:35387 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
Aug 07 04:23:36 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 04:23:36 dist-test-slave-zkpd krb5kdc[10433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.244.194@KRBTEST.COM for oryx/127.1.244.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 04:23:36.114065 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:36.104943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35625 (local address 127.1.244.194:56241)
0807 04:23:36.105523 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:36.105538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:36.105645 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:36.105946 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:36.105949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:36.106307 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:36.106508 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:36.106518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.107463 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.107467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:36.108407 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:36.108419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.109086 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.109090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:36.109219 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:36.109795 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:36.109813 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:36.113676 (+ 3863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.244.254:35625: Server oryx/127.1.244.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":278,"threads_started":1}
W20260807 04:23:36.114171 11473 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:36.105051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:56241 (local address 127.1.244.254:35625)
0807 04:23:36.105271 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:36.105275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:36.105609 (+ 334us) server_negotiation.cc:407] Connection header received
0807 04:23:36.105745 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:36.105749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:36.105809 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:36.105892 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:36.106630 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.107343 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.108526 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.108965 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.109057 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:36.113889 (+ 4832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.244.194:56241: BlockingRecv error: recv got EOF from 127.1.244.194:56241 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:23:36.511436 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11039
I20260807 04:23:36.523399 11518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:36.512518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.1.244.193:34291)
0807 04:23:36.513223 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:36.513229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:36.513255 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:23:36.513318 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:36.513325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:36.513557 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:36.513752 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:36.515087 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.516260 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.517438 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.517963 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.518066 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:36.520434 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:36.520443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:36.520455 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:36.520489 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:36.522903 (+ 2414us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.244.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 04:23:36.523192 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36640: Request ticket server kudu/127.1.244.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":351,"threads_started":1}
W20260807 04:23:36.523522 11518 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36640: Request ticket server kudu/127.1.244.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 04:23:36.523576 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11039: Not authorized: Client connection negotiation failed: client connection to 127.1.244.193:34291: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.244.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 04:23:36.523825 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11039
I20260807 04:23:36.544034 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11195
I20260807 04:23:36.556758 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:36.545181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.1.244.194:45083)
0807 04:23:36.545963 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:36.545972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:36.546007 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:23:36.546089 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:36.546100 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:36.546417 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:36.546626 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:36.547991 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.549097 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.550259 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.550811 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.550939 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:36.553488 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:36.553497 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:36.553511 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:36.553550 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:36.556168 (+ 2618us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.244.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 04:23:36.556456 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37262: Request ticket server kudu/127.1.244.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":398,"threads_started":1}
W20260807 04:23:36.556852 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11195: Not authorized: Client connection negotiation failed: client connection to 127.1.244.194:45083: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.244.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 04:23:36.557006 11519 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37262: Request ticket server kudu/127.1.244.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 04:23:36.557046 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11195
I20260807 04:23:36.575340 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11333
I20260807 04:23:36.587618 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:36.576441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.1.244.195:40459)
0807 04:23:36.577169 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:36.577178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:36.577209 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:23:36.577284 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:36.577292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:36.577617 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:36.577829 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:36.579069 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.580543 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.581652 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:36.582176 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:36.582302 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:36.584733 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:36.584742 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:36.584754 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:36.584790 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:36.587049 (+ 2259us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.244.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 04:23:36.587322 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34208: Request ticket server kudu/127.1.244.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":355,"threads_started":1}
W20260807 04:23:36.587630 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11333: Not authorized: Client connection negotiation failed: client connection to 127.1.244.195:40459: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.244.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 04:23:36.587762 11520 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34208: Request ticket server kudu/127.1.244.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 04:23:36.587800 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11333
I20260807 04:23:36.606026 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 10966
I20260807 04:23:36.607355 11032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:36.728031 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 10966
2026-08-07T04:23:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21533 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T04:23:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:36Z Disabled control of system clock
I20260807 04:23:36.810535 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46775
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 04:23:37.143244 11525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:37.143522 11525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:37.143603 11525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:37.150673 11525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:37.150774 11525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:37.150888 11525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:37.150977 11525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:37.151036 11525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:37.159965 11525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:37.161594 11525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:37.164462 11525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:37.178567 11530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:37.178610 11531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:37.179204 11533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:37.179872 11525 server_base.cc:1034] running on GCE node
I20260807 04:23:37.180927 11525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:37.182881 11525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:37.184141 11525 hybrid_clock.cc:648] HybridClock initialized: now 1786076617184082 us; error 39 us; skew 500 ppm
I20260807 04:23:37.184638 11525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:37.187633 11525 webserver.cc:460] Webserver started at http://127.1.244.254:33683/ using document root <none> and password file <none>
I20260807 04:23:37.188612 11525 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:37.188750 11525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:37.189154 11525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:37.193650 11525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "9e0e30f53c684817b275c4309d59d6e7"
format_stamp: "Formatted at 2026-08-07 04:23:37 on dist-test-slave-zkpd"
server_key: "28de77ba9748b3a9ffea9346d434c0ed"
server_key_iv: "00f25c19c6757dbb6ab3436de46739c6"
server_key_version: "encryptionkey@0"
I20260807 04:23:37.194573 11525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "9e0e30f53c684817b275c4309d59d6e7"
format_stamp: "Formatted at 2026-08-07 04:23:37 on dist-test-slave-zkpd"
server_key: "28de77ba9748b3a9ffea9346d434c0ed"
server_key_iv: "00f25c19c6757dbb6ab3436de46739c6"
server_key_version: "encryptionkey@0"
I20260807 04:23:37.202019 11525 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 04:23:37.206033 11539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:37.207736 11525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:23:37.208029 11525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "9e0e30f53c684817b275c4309d59d6e7"
format_stamp: "Formatted at 2026-08-07 04:23:37 on dist-test-slave-zkpd"
server_key: "28de77ba9748b3a9ffea9346d434c0ed"
server_key_iv: "00f25c19c6757dbb6ab3436de46739c6"
server_key_version: "encryptionkey@0"
I20260807 04:23:37.208272 11525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:37.224252 11525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:37.224999 11525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:37.225279 11525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:37.243221 11525 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46775
I20260807 04:23:37.243243 11592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46775 every 8 connection(s)
I20260807 04:23:37.244563 11525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:23:37.247983 11593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:37.252491 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11525
I20260807 04:23:37.252686 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:23:37.253539 2003 external_mini_cluster.cc:1428] Setting key 02f45d90bd629983d5c0b96cfe1eeac7
I20260807 04:23:37.259057 11593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7: Bootstrap starting.
I20260807 04:23:37.262055 11593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:37.263049 11593 log.cc:824] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:37.266052 11593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7: No bootstrap required, opened a new log
I20260807 04:23:37.266820 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:37.255574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50558 (local address 127.1.244.254:46775)
0807 04:23:37.256557 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:37.256570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:37.256611 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:23:37.258478 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:37.258499 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:37.258922 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:37.259288 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:37.259883 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:37.261191 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:37.262579 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:37.263256 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:37.263439 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:37.264727 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:37.264734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:37.264747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:37.265242 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:37.265323 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:37.265343 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:37.265429 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:37.265661 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 04:23:37.265971 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":391,"threads_started":1}
I20260807 04:23:37.274272 11593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0e30f53c684817b275c4309d59d6e7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46775 } }
I20260807 04:23:37.274597 11593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:37.274709 11593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0e30f53c684817b275c4309d59d6e7, State: Initialized, Role: FOLLOWER
I20260807 04:23:37.275324 11593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0e30f53c684817b275c4309d59d6e7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46775 } }
I20260807 04:23:37.275549 11593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:23:37.275674 11593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:23:37.275859 11593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:37.278028 11593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0e30f53c684817b275c4309d59d6e7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46775 } }
I20260807 04:23:37.278537 11593 leader_election.cc:302] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e0e30f53c684817b275c4309d59d6e7; no voters:
I20260807 04:23:37.279176 11593 leader_election.cc:288] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:23:37.279273 11598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:37.280061 11598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [term 1 LEADER]: Becoming Leader. State: Replica: 9e0e30f53c684817b275c4309d59d6e7, State: Running, Role: LEADER
I20260807 04:23:37.280526 11598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0e30f53c684817b275c4309d59d6e7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46775 } }
I20260807 04:23:37.281242 11593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:37.283586 11601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0e30f53c684817b275c4309d59d6e7. Latest consensus state: current_term: 1 leader_uuid: "9e0e30f53c684817b275c4309d59d6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0e30f53c684817b275c4309d59d6e7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46775 } } }
I20260807 04:23:37.283667 11600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e0e30f53c684817b275c4309d59d6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0e30f53c684817b275c4309d59d6e7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46775 } } }
I20260807 04:23:37.283963 11601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:37.284078 11600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:37.285445 11606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:37.289386 11606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:37.297670 11606 catalog_manager.cc:1347] Generated new cluster ID: a425b55b68104092a3f20cfdf3e17709
I20260807 04:23:37.297780 11606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:37.307505 11606 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:23:37.308881 11606 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:37.320659 11606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e0e30f53c684817b275c4309d59d6e7: Generated new TSK 0
I20260807 04:23:37.321460 11606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:23:37.326162 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46775
--builtin_ntp_servers=127.1.244.212:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 04:23:37.705493 11618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:37.705766 11618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:37.705940 11618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:37.713744 11618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:37.713850 11618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:37.714046 11618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:37.723384 11618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:37.724722 11618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:37.726570 11618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:37.739667 11623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:37.739676 11624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:37.739718 11626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:37.745342 11618 server_base.cc:1034] running on GCE node
I20260807 04:23:37.746006 11618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:37.747057 11618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:37.748340 11618 hybrid_clock.cc:648] HybridClock initialized: now 1786076617748299 us; error 45 us; skew 500 ppm
I20260807 04:23:37.748713 11618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:37.751003 11618 webserver.cc:460] Webserver started at http://127.1.244.193:45631/ using document root <none> and password file <none>
I20260807 04:23:37.751735 11618 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:37.751879 11618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:37.752259 11618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:37.755157 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "453105d5322c43ccb1114676832258c0"
format_stamp: "Formatted at 2026-08-07 04:23:37 on dist-test-slave-zkpd"
server_key: "e1ff6ff36af02d2b64ee052af499129c"
server_key_iv: "945c05c3048e96b73022f32ca32c00e5"
server_key_version: "encryptionkey@0"
I20260807 04:23:37.756053 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "453105d5322c43ccb1114676832258c0"
format_stamp: "Formatted at 2026-08-07 04:23:37 on dist-test-slave-zkpd"
server_key: "e1ff6ff36af02d2b64ee052af499129c"
server_key_iv: "945c05c3048e96b73022f32ca32c00e5"
server_key_version: "encryptionkey@0"
I20260807 04:23:37.761157 11618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:23:37.764727 11632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:37.766004 11618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:23:37.766187 11618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "453105d5322c43ccb1114676832258c0"
format_stamp: "Formatted at 2026-08-07 04:23:37 on dist-test-slave-zkpd"
server_key: "e1ff6ff36af02d2b64ee052af499129c"
server_key_iv: "945c05c3048e96b73022f32ca32c00e5"
server_key_version: "encryptionkey@0"
I20260807 04:23:37.766412 11618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:37.784008 11618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:37.784953 11618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:37.785395 11618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:37.786707 11618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:37.788278 11618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:37.788378 11618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:37.788470 11618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:37.788532 11618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:37.823611 11618 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:32809
I20260807 04:23:37.823663 11748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:32809 every 8 connection(s)
I20260807 04:23:37.825028 11618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:37.828212 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11618
I20260807 04:23:37.828425 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:23:37.829092 2003 external_mini_cluster.cc:1428] Setting key cbd545d940da07014ec42f00deb338b6
I20260807 04:23:37.831733 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46775
--builtin_ntp_servers=127.1.244.212:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 04:23:37.839021 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:37.827649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:48331 (local address 127.1.244.254:46775)
0807 04:23:37.828227 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:37.828235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:37.829794 (+ 1559us) server_negotiation.cc:407] Connection header received
0807 04:23:37.831677 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:37.831683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:37.831778 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:37.831941 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:37.833223 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:37.834036 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:37.835807 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:37.836316 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:37.836461 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:37.838049 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:37.838073 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:37.838077 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:37.838096 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:37.838141 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:37.838154 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:37.838265 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:37.838522 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 04:23:37.838631 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":275,"threads_started":1}
I20260807 04:23:37.840162 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:37.828404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46775 (local address 127.1.244.193:48331)
0807 04:23:37.829584 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:37.829641 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:37.831390 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:37.832335 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:37.832345 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:37.832433 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:37.833032 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:37.833051 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:37.834228 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:37.834234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:37.835614 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:37.835628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:37.836570 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:37.836578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:37.836855 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:37.837690 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:23:37.837694 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:37.837740 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:23:37.837818 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:37.838267 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:37.838282 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:37.838293 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 04:23:37.838510 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 04:23:37.838915 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":385,"threads_started":1}
I20260807 04:23:37.841837 11749 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46775
I20260807 04:23:37.842164 11749 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:37.843031 11749 heartbeater.cc:511] Master 127.1.244.254:46775 requested a full tablet report, sending...
I20260807 04:23:37.845402 11557 ts_manager.cc:194] Registered new tserver with Master: 453105d5322c43ccb1114676832258c0 (127.1.244.193:32809)
I20260807 04:23:37.847564 11557 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:48331
W20260807 04:23:38.165742 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:38.165992 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:38.166074 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:38.173933 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:38.174022 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:38.174127 11753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:38.183123 11753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:38.184391 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:38.186115 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:38.199666 11760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:38.199702 11759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:38.199735 11762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:38.200477 11753 server_base.cc:1034] running on GCE node
I20260807 04:23:38.201030 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:38.202262 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:38.203508 11753 hybrid_clock.cc:648] HybridClock initialized: now 1786076618203476 us; error 50 us; skew 500 ppm
I20260807 04:23:38.203816 11753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:38.206094 11753 webserver.cc:460] Webserver started at http://127.1.244.194:42935/ using document root <none> and password file <none>
I20260807 04:23:38.206761 11753 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:38.206894 11753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:38.207217 11753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:38.210070 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "4b0691b5df564c8d8d718a0adfe9b2af"
format_stamp: "Formatted at 2026-08-07 04:23:38 on dist-test-slave-zkpd"
server_key: "eee8e1023aabb0e30abf011323ac1929"
server_key_iv: "67c49f3991927b6e2b6b1291afcf24a4"
server_key_version: "encryptionkey@0"
I20260807 04:23:38.210858 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "4b0691b5df564c8d8d718a0adfe9b2af"
format_stamp: "Formatted at 2026-08-07 04:23:38 on dist-test-slave-zkpd"
server_key: "eee8e1023aabb0e30abf011323ac1929"
server_key_iv: "67c49f3991927b6e2b6b1291afcf24a4"
server_key_version: "encryptionkey@0"
I20260807 04:23:38.215866 11753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:23:38.219089 11768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:38.220548 11753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:38.220748 11753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "4b0691b5df564c8d8d718a0adfe9b2af"
format_stamp: "Formatted at 2026-08-07 04:23:38 on dist-test-slave-zkpd"
server_key: "eee8e1023aabb0e30abf011323ac1929"
server_key_iv: "67c49f3991927b6e2b6b1291afcf24a4"
server_key_version: "encryptionkey@0"
I20260807 04:23:38.220973 11753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:38.256955 11753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:38.257764 11753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:38.258068 11753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:38.258984 11753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:38.260433 11753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:38.260557 11753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:38.260697 11753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:38.260788 11753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:38.294780 11753 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:36705
I20260807 04:23:38.294834 11889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:36705 every 8 connection(s)
I20260807 04:23:38.296108 11753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:38.301379 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11753
I20260807 04:23:38.301538 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:23:38.302021 2003 external_mini_cluster.cc:1428] Setting key c4c2cb2810819ac920952b3909863303
I20260807 04:23:38.305145 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46775
--builtin_ntp_servers=127.1.244.212:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 04:23:38.309494 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.298609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:39103 (local address 127.1.244.254:46775)
0807 04:23:38.298819 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:38.298825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:38.300332 (+ 1507us) server_negotiation.cc:407] Connection header received
0807 04:23:38.302179 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:38.302185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:38.302271 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:38.302380 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:38.303687 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.304762 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.306438 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.307151 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.307295 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:38.308603 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:38.308612 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:38.308619 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:38.308643 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:38.308683 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:38.308701 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:38.308796 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:38.308986 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 04:23:38.309090 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:23:38.310303 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.299208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46775 (local address 127.1.244.194:39103)
0807 04:23:38.300178 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:38.300228 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:38.301886 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:38.302608 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:38.302621 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:38.302762 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:38.303514 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:38.303539 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.304914 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.304918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:38.306283 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:38.306295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.307350 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.307355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:38.307543 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:38.308306 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:23:38.308308 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:38.308347 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:23:38.308422 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:38.308798 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:38.308805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:38.308811 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:23:38.308945 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 04:23:38.309235 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":277,"threads_started":1}
I20260807 04:23:38.311659 11890 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46775
I20260807 04:23:38.312006 11890 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:38.312885 11890 heartbeater.cc:511] Master 127.1.244.254:46775 requested a full tablet report, sending...
I20260807 04:23:38.314349 11557 ts_manager.cc:194] Registered new tserver with Master: 4b0691b5df564c8d8d718a0adfe9b2af (127.1.244.194:36705)
I20260807 04:23:38.315119 11557 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:39103
W20260807 04:23:38.634217 11893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:38.634482 11893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:38.634629 11893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:38.641762 11893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:38.641850 11893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:38.642045 11893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:38.650874 11893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:46775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:38.652170 11893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:38.653981 11893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:38.667042 11901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:38.667117 11899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:38.667305 11898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:38.668033 11893 server_base.cc:1034] running on GCE node
I20260807 04:23:38.668622 11893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:38.669507 11893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:38.670727 11893 hybrid_clock.cc:648] HybridClock initialized: now 1786076618670643 us; error 55 us; skew 500 ppm
I20260807 04:23:38.671068 11893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:38.673280 11893 webserver.cc:460] Webserver started at http://127.1.244.195:41653/ using document root <none> and password file <none>
I20260807 04:23:38.673971 11893 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:38.674091 11893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:38.674420 11893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:38.677301 11893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "655de8c5f9294d59a4fd43798fcde5be"
format_stamp: "Formatted at 2026-08-07 04:23:38 on dist-test-slave-zkpd"
server_key: "085c15192c378754899e8421f1591814"
server_key_iv: "385f4c89febc2458d39cb2229905af44"
server_key_version: "encryptionkey@0"
I20260807 04:23:38.678089 11893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "655de8c5f9294d59a4fd43798fcde5be"
format_stamp: "Formatted at 2026-08-07 04:23:38 on dist-test-slave-zkpd"
server_key: "085c15192c378754899e8421f1591814"
server_key_iv: "385f4c89febc2458d39cb2229905af44"
server_key_version: "encryptionkey@0"
I20260807 04:23:38.683226 11893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:23:38.686587 11907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:38.688037 11893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:23:38.688197 11893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "655de8c5f9294d59a4fd43798fcde5be"
format_stamp: "Formatted at 2026-08-07 04:23:38 on dist-test-slave-zkpd"
server_key: "085c15192c378754899e8421f1591814"
server_key_iv: "385f4c89febc2458d39cb2229905af44"
server_key_version: "encryptionkey@0"
I20260807 04:23:38.688421 11893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:38.702952 11893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:38.703763 11893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:38.704135 11893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:38.705088 11893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:38.706465 11893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:38.706549 11893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:38.706707 11893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:38.706799 11893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:38.740126 11893 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:36663
I20260807 04:23:38.740199 12022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:36663 every 8 connection(s)
I20260807 04:23:38.741416 11893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:38.741902 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 11893
I20260807 04:23:38.742111 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:23:38.742777 2003 external_mini_cluster.cc:1428] Setting key 22763f33061dad7ea3b4ae0bdb73323e
I20260807 04:23:38.753862 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.744156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:48451 (local address 127.1.244.254:46775)
0807 04:23:38.744349 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:38.744355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:38.745877 (+ 1522us) server_negotiation.cc:407] Connection header received
0807 04:23:38.747630 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:38.747635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:38.747727 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:38.747835 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:38.748860 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.749616 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.751138 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.751559 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.751688 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:38.752993 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:38.753007 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:38.753011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:38.753027 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:38.753056 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:38.753068 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:38.753162 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:38.753397 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 04:23:38.753494 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:23:38.754766 12025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.744754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:46775 (local address 127.1.244.195:48451)
0807 04:23:38.745735 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:38.745775 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:38.747385 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:38.748089 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:38.748097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:38.748175 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:38.748705 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:38.748720 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.749771 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.749776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:38.750986 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:38.750998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.751785 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.751792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:38.752082 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:38.752687 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:23:38.752690 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:38.752726 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:23:38.752791 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:38.753161 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:38.753173 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:38.753183 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:23:38.753360 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 04:23:38.753723 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":355,"threads_started":1}
I20260807 04:23:38.756130 12023 heartbeater.cc:347] Connected to a master server at 127.1.244.254:46775
I20260807 04:23:38.756438 12023 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:38.757376 12023 heartbeater.cc:511] Master 127.1.244.254:46775 requested a full tablet report, sending...
I20260807 04:23:38.758965 11557 ts_manager.cc:194] Registered new tserver with Master: 655de8c5f9294d59a4fd43798fcde5be (127.1.244.195:36663)
I20260807 04:23:38.759734 11557 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:48451
I20260807 04:23:38.770287 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 04:23:38.776635 11751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:38.774991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.1.244.254:46775)
0807 04:23:38.775138 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:38.775142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:38.775706 (+ 564us) server_negotiation.cc:407] Connection header received
0807 04:23:38.775816 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:38.775820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:38.775939 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:38.776030 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:38.776427 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50560: BlockingRecv error: recv got EOF from 127.0.0.1:50560 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:23:38.777578 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11618
I20260807 04:23:38.784859 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.778464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.1.244.193:32809)
0807 04:23:38.778970 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:38.778976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:38.779000 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:23:38.779062 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:38.779068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:38.779247 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:38.779370 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:38.779675 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.780728 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.781967 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.782571 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.782666 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:38.783578 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:38.783587 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:38.783597 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:38.784198 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:38.784244 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:38.784269 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:38.784357 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:38.784458 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 04:23:38.784573 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":273,"threads_started":1}
I20260807 04:23:38.785982 11744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:38.866253 11749 heartbeater.cc:503] Master 127.1.244.254:46775 was elected leader, sending a full tablet report...
I20260807 04:23:38.867635 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11618
W20260807 04:23:38.886461 11542 connection.cc:535] server connection from 127.1.244.193:48331 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 04:23:38.886828 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11753
I20260807 04:23:38.894657 12034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.887680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.1.244.194:36705)
0807 04:23:38.888256 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:38.888263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:38.888289 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:23:38.888345 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:38.888350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:38.888563 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:38.888685 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:38.889049 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.890054 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.891333 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.892046 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.892138 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:38.893155 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:38.893163 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:38.893176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:38.893833 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:38.893878 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:38.893897 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:38.894023 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:38.894184 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 04:23:38.894311 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":291,"threads_started":1}
I20260807 04:23:38.895876 11885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:38.975152 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11753
I20260807 04:23:38.993848 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11893
I20260807 04:23:39.001488 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:38.994847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.1.244.195:36663)
0807 04:23:38.995527 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:38.995534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:38.995565 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:23:38.995628 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:38.995635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:38.995836 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:38.996042 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:38.996397 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.997374 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.998488 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:38.999037 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:38.999129 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:39.000181 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:39.000189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:39.000200 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:39.000748 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:39.000789 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:39.000812 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:39.000927 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:39.001057 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 04:23:39.001171 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":366,"threads_started":1}
I20260807 04:23:39.002676 12018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:39.083869 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11893
I20260807 04:23:39.101841 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 11525
I20260807 04:23:39.103139 11588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:39.203894 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 11525
2026-08-07T04:23:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2474 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T04:23:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:39Z Disabled control of system clock
I20260807 04:23:39.283347 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34067
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:34067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 04:23:39.612363 12042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:39.612612 12042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:39.612703 12042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:39.620323 12042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:39.620428 12042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:39.620466 12042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:39.620522 12042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:39.620568 12042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:39.629402 12042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:34067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:39.630663 12042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:39.632423 12042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:39.644229 12050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:39.644263 12047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:39.644601 12048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:39.645354 12042 server_base.cc:1034] running on GCE node
I20260807 04:23:39.646090 12042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:39.647516 12042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:39.648770 12042 hybrid_clock.cc:648] HybridClock initialized: now 1786076619648732 us; error 47 us; skew 500 ppm
I20260807 04:23:39.649107 12042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:39.651299 12042 webserver.cc:460] Webserver started at http://127.1.244.254:42873/ using document root <none> and password file <none>
I20260807 04:23:39.652038 12042 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:39.652171 12042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:39.652503 12042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:39.656986 12042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "41fcf5bda65b42419dcdcc477d94aafa"
format_stamp: "Formatted at 2026-08-07 04:23:39 on dist-test-slave-zkpd"
server_key: "857dc5679aa042c17043d262cc7420db"
server_key_iv: "8406614b6892aed4ab0aac58ee2847be"
server_key_version: "encryptionkey@0"
I20260807 04:23:39.657800 12042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "41fcf5bda65b42419dcdcc477d94aafa"
format_stamp: "Formatted at 2026-08-07 04:23:39 on dist-test-slave-zkpd"
server_key: "857dc5679aa042c17043d262cc7420db"
server_key_iv: "8406614b6892aed4ab0aac58ee2847be"
server_key_version: "encryptionkey@0"
I20260807 04:23:39.662662 12042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:23:39.665995 12056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:39.667138 12042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 04:23:39.667318 12042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "41fcf5bda65b42419dcdcc477d94aafa"
format_stamp: "Formatted at 2026-08-07 04:23:39 on dist-test-slave-zkpd"
server_key: "857dc5679aa042c17043d262cc7420db"
server_key_iv: "8406614b6892aed4ab0aac58ee2847be"
server_key_version: "encryptionkey@0"
I20260807 04:23:39.667529 12042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:39.672945 12042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:39.673744 12042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:39.674043 12042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:39.692157 12042 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:34067
I20260807 04:23:39.692194 12109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:34067 every 8 connection(s)
I20260807 04:23:39.693436 12042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:23:39.693924 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12042
I20260807 04:23:39.694160 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:23:39.695011 2003 external_mini_cluster.cc:1428] Setting key af57ef4db08a68eb5a69f848e65e0af1
I20260807 04:23:39.697868 12110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:39.704483 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:39.697027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.1.244.254:34067)
0807 04:23:39.698155 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:39.698170 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:39.698220 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:23:39.700501 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:39.700526 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:39.701054 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:39.701480 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:39.702509 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:39.702518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:39.702534 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:39.702550 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:39.702642 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:39.702662 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:39.702889 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:39.703042 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 04:23:39.703291 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":344,"threads_started":1}
I20260807 04:23:39.710582 12110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa: Bootstrap starting.
I20260807 04:23:39.713501 12110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:39.714463 12110 log.cc:824] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:39.717095 12110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa: No bootstrap required, opened a new log
I20260807 04:23:39.724009 12110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fcf5bda65b42419dcdcc477d94aafa" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34067 } }
I20260807 04:23:39.724315 12110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:39.724386 12110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41fcf5bda65b42419dcdcc477d94aafa, State: Initialized, Role: FOLLOWER
I20260807 04:23:39.724843 12110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fcf5bda65b42419dcdcc477d94aafa" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34067 } }
I20260807 04:23:39.725041 12110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:23:39.725133 12110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:23:39.725258 12110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:39.727171 12110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fcf5bda65b42419dcdcc477d94aafa" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34067 } }
I20260807 04:23:39.727591 12110 leader_election.cc:302] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41fcf5bda65b42419dcdcc477d94aafa; no voters:
I20260807 04:23:39.728118 12110 leader_election.cc:288] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:23:39.728281 12115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:39.728967 12115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [term 1 LEADER]: Becoming Leader. State: Replica: 41fcf5bda65b42419dcdcc477d94aafa, State: Running, Role: LEADER
I20260807 04:23:39.729498 12115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fcf5bda65b42419dcdcc477d94aafa" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34067 } }
I20260807 04:23:39.730192 12110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:39.732599 12117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41fcf5bda65b42419dcdcc477d94aafa. Latest consensus state: current_term: 1 leader_uuid: "41fcf5bda65b42419dcdcc477d94aafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fcf5bda65b42419dcdcc477d94aafa" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34067 } } }
I20260807 04:23:39.732623 12116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41fcf5bda65b42419dcdcc477d94aafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fcf5bda65b42419dcdcc477d94aafa" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34067 } } }
I20260807 04:23:39.733268 12116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:39.733270 12117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:39.734911 12124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:39.739993 12124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:39.750496 12124 catalog_manager.cc:1347] Generated new cluster ID: 612e2a3bea2e49228c9a14667fe42389
I20260807 04:23:39.750617 12124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:39.764786 12124 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:23:39.765172 12124 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:39.771377 12124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41fcf5bda65b42419dcdcc477d94aafa: Generated new TSK 0
I20260807 04:23:39.772181 12124 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:23:39.779134 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34067
--builtin_ntp_servers=127.1.244.212:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 04:23:40.137944 12136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:40.138234 12136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:40.138348 12136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:40.145418 12136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:40.145506 12136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:40.145685 12136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:40.154510 12136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:40.155778 12136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:40.157582 12136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:40.170639 12143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:40.170791 12142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:40.170764 12145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:40.171687 12136 server_base.cc:1034] running on GCE node
I20260807 04:23:40.172396 12136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:40.173247 12136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:40.174453 12136 hybrid_clock.cc:648] HybridClock initialized: now 1786076620174423 us; error 29 us; skew 500 ppm
I20260807 04:23:40.174724 12136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:40.176868 12136 webserver.cc:460] Webserver started at http://127.1.244.193:33035/ using document root <none> and password file <none>
I20260807 04:23:40.177554 12136 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:40.177668 12136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:40.178009 12136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:40.180943 12136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "7501c81bc11f49019858eff1bae9afb1"
format_stamp: "Formatted at 2026-08-07 04:23:40 on dist-test-slave-zkpd"
server_key: "64d2d898e3de5396c16c74857c1ec04f"
server_key_iv: "292c5e652000463055a6d6f245fb1c53"
server_key_version: "encryptionkey@0"
I20260807 04:23:40.181703 12136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "7501c81bc11f49019858eff1bae9afb1"
format_stamp: "Formatted at 2026-08-07 04:23:40 on dist-test-slave-zkpd"
server_key: "64d2d898e3de5396c16c74857c1ec04f"
server_key_iv: "292c5e652000463055a6d6f245fb1c53"
server_key_version: "encryptionkey@0"
I20260807 04:23:40.186492 12136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:23:40.189769 12152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:40.190989 12136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 04:23:40.191160 12136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "7501c81bc11f49019858eff1bae9afb1"
format_stamp: "Formatted at 2026-08-07 04:23:40 on dist-test-slave-zkpd"
server_key: "64d2d898e3de5396c16c74857c1ec04f"
server_key_iv: "292c5e652000463055a6d6f245fb1c53"
server_key_version: "encryptionkey@0"
I20260807 04:23:40.191385 12136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:40.196959 12136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:40.197706 12136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:40.197973 12136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:40.198966 12136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:40.200387 12136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:40.200516 12136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:40.200654 12136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:40.200742 12136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:40.234302 12136 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44889
I20260807 04:23:40.234395 12272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44889 every 8 connection(s)
I20260807 04:23:40.235895 12136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:40.239455 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12136
I20260807 04:23:40.239678 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:23:40.240509 2003 external_mini_cluster.cc:1428] Setting key 4ef8f2b2c9f479bceb465eaf5634ea65
I20260807 04:23:40.243539 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34067
--builtin_ntp_servers=127.1.244.212:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 04:23:40.244872 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:40.238397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:40785 (local address 127.1.244.254:34067)
0807 04:23:40.238809 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:40.238814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:40.240334 (+ 1520us) server_negotiation.cc:407] Connection header received
0807 04:23:40.242483 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:40.242488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:40.242566 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:40.242682 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:40.244107 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:40.244113 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:40.244118 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:40.244135 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:40.244184 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:40.244195 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:40.244317 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:40.244517 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 04:23:40.244584 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":220,"threads_started":1}
I20260807 04:23:40.245858 12276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:40.238993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34067 (local address 127.1.244.193:40785)
0807 04:23:40.240155 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:40.240209 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:40.242207 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:40.242915 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:40.242923 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:40.243020 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:40.243737 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:23:40.243741 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:40.243788 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:23:40.243941 (+ 153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:40.244293 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:40.244305 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:40.244313 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 04:23:40.244596 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 04:23:40.244830 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":351,"threads_started":1}
I20260807 04:23:40.247336 12273 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34067
I20260807 04:23:40.247608 12273 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:40.248210 12273 heartbeater.cc:511] Master 127.1.244.254:34067 requested a full tablet report, sending...
I20260807 04:23:40.250396 12073 ts_manager.cc:194] Registered new tserver with Master: 7501c81bc11f49019858eff1bae9afb1 (127.1.244.193:44889)
W20260807 04:23:40.575552 12277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:40.575812 12277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:40.576025 12277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:40.583043 12277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:40.583142 12277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:40.583280 12277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:40.591777 12277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:40.592998 12277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:40.594801 12277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:40.607605 12284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:40.607688 12283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:40.607657 12286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:40.608511 12277 server_base.cc:1034] running on GCE node
I20260807 04:23:40.609113 12277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:40.609928 12277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:40.611123 12277 hybrid_clock.cc:648] HybridClock initialized: now 1786076620611087 us; error 34 us; skew 500 ppm
I20260807 04:23:40.611410 12277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:40.613504 12277 webserver.cc:460] Webserver started at http://127.1.244.194:41891/ using document root <none> and password file <none>
I20260807 04:23:40.614167 12277 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:40.614252 12277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:40.614516 12277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:40.617220 12277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "642d6fbd6b924b71ae7a4c009cfae521"
format_stamp: "Formatted at 2026-08-07 04:23:40 on dist-test-slave-zkpd"
server_key: "c9295b5cfb1b21835968f3b0c6785b9e"
server_key_iv: "5244cf60eb4a9b796fbbff3c30ee2a3a"
server_key_version: "encryptionkey@0"
I20260807 04:23:40.617971 12277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "642d6fbd6b924b71ae7a4c009cfae521"
format_stamp: "Formatted at 2026-08-07 04:23:40 on dist-test-slave-zkpd"
server_key: "c9295b5cfb1b21835968f3b0c6785b9e"
server_key_iv: "5244cf60eb4a9b796fbbff3c30ee2a3a"
server_key_version: "encryptionkey@0"
I20260807 04:23:40.622843 12277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 04:23:40.626200 12292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:40.627499 12277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:23:40.627636 12277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "642d6fbd6b924b71ae7a4c009cfae521"
format_stamp: "Formatted at 2026-08-07 04:23:40 on dist-test-slave-zkpd"
server_key: "c9295b5cfb1b21835968f3b0c6785b9e"
server_key_iv: "5244cf60eb4a9b796fbbff3c30ee2a3a"
server_key_version: "encryptionkey@0"
I20260807 04:23:40.627826 12277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:40.633924 12277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:40.634686 12277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:40.634965 12277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:40.636013 12277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:40.637368 12277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:40.637482 12277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:40.637614 12277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:40.637710 12277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:40.670085 12277 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:46279
I20260807 04:23:40.670101 12407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:46279 every 8 connection(s)
I20260807 04:23:40.671475 12277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:40.679266 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:40.674084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:57505 (local address 127.1.244.254:34067)
0807 04:23:40.674271 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:40.674275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:40.675583 (+ 1308us) server_negotiation.cc:407] Connection header received
0807 04:23:40.677169 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:40.677173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:40.677223 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:40.677314 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:40.678668 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:40.678672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:40.678675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:40.678690 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:40.678722 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:40.678734 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:40.678837 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:40.678996 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 04:23:40.679045 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:23:40.680194 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:40.674587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34067 (local address 127.1.244.194:57505)
0807 04:23:40.675440 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:40.675472 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:40.676946 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:40.677494 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:40.677502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:40.677645 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:40.678367 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:23:40.678369 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:40.678412 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:23:40.678493 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:40.678818 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:40.678825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:40.678830 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:23:40.679039 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 04:23:40.679212 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":282,"threads_started":1}
I20260807 04:23:40.680447 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12277
I20260807 04:23:40.680648 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:23:40.681254 2003 external_mini_cluster.cc:1428] Setting key e3037176d1310ba97342d99aec5271b4
I20260807 04:23:40.681599 12408 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34067
I20260807 04:23:40.681908 12408 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:40.682476 12408 heartbeater.cc:511] Master 127.1.244.254:34067 requested a full tablet report, sending...
I20260807 04:23:40.683871 12073 ts_manager.cc:194] Registered new tserver with Master: 642d6fbd6b924b71ae7a4c009cfae521 (127.1.244.194:46279)
I20260807 04:23:40.684252 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34067
--builtin_ntp_servers=127.1.244.212:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 04:23:41.010293 12411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:41.010552 12411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:41.010636 12411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:41.017601 12411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:41.017691 12411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:41.017796 12411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:41.026540 12411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:34067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:41.027707 12411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:41.029489 12411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:41.042760 12416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:41.042760 12419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:41.042743 12417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:41.043828 12411 server_base.cc:1034] running on GCE node
I20260807 04:23:41.044409 12411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:41.045230 12411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:41.046428 12411 hybrid_clock.cc:648] HybridClock initialized: now 1786076621046393 us; error 25 us; skew 500 ppm
I20260807 04:23:41.046689 12411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:23:41.048866 12411 webserver.cc:460] Webserver started at http://127.1.244.195:36687/ using document root <none> and password file <none>
I20260807 04:23:41.049559 12411 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:41.049670 12411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:41.050011 12411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:41.052863 12411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "e8638ce825d44f8ea86e71f08d331157"
format_stamp: "Formatted at 2026-08-07 04:23:41 on dist-test-slave-zkpd"
server_key: "b02539072558f15ac42e7c7cdd6231db"
server_key_iv: "835404f859b6c4c0da9152281774ba79"
server_key_version: "encryptionkey@0"
I20260807 04:23:41.053648 12411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "e8638ce825d44f8ea86e71f08d331157"
format_stamp: "Formatted at 2026-08-07 04:23:41 on dist-test-slave-zkpd"
server_key: "b02539072558f15ac42e7c7cdd6231db"
server_key_iv: "835404f859b6c4c0da9152281774ba79"
server_key_version: "encryptionkey@0"
I20260807 04:23:41.058575 12411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:23:41.061867 12425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:41.063026 12411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:23:41.063196 12411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "e8638ce825d44f8ea86e71f08d331157"
format_stamp: "Formatted at 2026-08-07 04:23:41 on dist-test-slave-zkpd"
server_key: "b02539072558f15ac42e7c7cdd6231db"
server_key_iv: "835404f859b6c4c0da9152281774ba79"
server_key_version: "encryptionkey@0"
I20260807 04:23:41.063403 12411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:41.068866 12411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:41.069617 12411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:41.069890 12411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:41.070737 12411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:41.072193 12411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:41.072310 12411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:41.072444 12411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:41.072551 12411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:41.105760 12411 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41555
I20260807 04:23:41.105804 12542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41555 every 8 connection(s)
I20260807 04:23:41.107110 12411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:41.110044 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12411
I20260807 04:23:41.110247 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:23:41.110714 2003 external_mini_cluster.cc:1428] Setting key 9a0f132d0f72db70ee045656f7481bf1
I20260807 04:23:41.115983 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:41.109721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:43199 (local address 127.1.244.254:34067)
0807 04:23:41.109967 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:41.109974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:41.111639 (+ 1665us) server_negotiation.cc:407] Connection header received
0807 04:23:41.113711 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:41.113716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:41.113818 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:41.113931 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:41.115175 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:41.115181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:41.115186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:41.115210 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:41.115246 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:41.115258 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:41.115391 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:41.115581 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 04:23:41.115650 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 04:23:41.116822 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:41.110381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:34067 (local address 127.1.244.195:43199)
0807 04:23:41.111466 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:41.111522 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:41.113395 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:41.114159 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:41.114167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:41.114260 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:41.114859 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:23:41.114861 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:41.114900 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:23:41.114982 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:41.115356 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:41.115369 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:41.115379 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 04:23:41.115667 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 04:23:41.115896 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":345,"threads_started":1}
I20260807 04:23:41.117959 12543 heartbeater.cc:347] Connected to a master server at 127.1.244.254:34067
I20260807 04:23:41.118239 12543 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:41.118783 12543 heartbeater.cc:511] Master 127.1.244.254:34067 requested a full tablet report, sending...
I20260807 04:23:41.120131 12073 ts_manager.cc:194] Registered new tserver with Master: e8638ce825d44f8ea86e71f08d331157 (127.1.244.195:41555)
I20260807 04:23:41.125878 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 04:23:41.132131 12275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:23:41.130285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.1.244.254:34067)
0807 04:23:41.130442 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:41.130446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:41.131074 (+ 628us) server_negotiation.cc:407] Connection header received
0807 04:23:41.131259 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:41.131265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:41.131327 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:41.131430 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:41.131840 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48654: BlockingRecv error: recv got EOF from 127.0.0.1:48654 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:23:41.132805 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12136
I20260807 04:23:41.136462 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:41.133830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.1.244.193:44889)
0807 04:23:41.134406 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:41.134415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:41.134448 (+ 33us) server_negotiation.cc:407] Connection header received
0807 04:23:41.134504 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:41.134509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:41.134686 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:41.134813 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:41.135717 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:41.135724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:41.135742 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:41.135759 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:41.135796 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:41.135809 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:41.135974 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:41.136057 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 04:23:41.136157 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":286,"threads_started":1}
I20260807 04:23:41.137594 12268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:41.217502 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12136
I20260807 04:23:41.236414 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12277
I20260807 04:23:41.239835 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:41.237326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.1.244.194:46279)
0807 04:23:41.237878 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:41.237885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:41.237917 (+ 32us) server_negotiation.cc:407] Connection header received
0807 04:23:41.237988 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:41.237994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:41.238212 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:41.238337 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:41.239236 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:41.239242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:41.239254 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:41.239266 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:41.239308 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:41.239322 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:41.239422 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:41.239518 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 04:23:41.239596 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":291,"threads_started":1}
I20260807 04:23:41.241047 12403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:41.321445 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12277
I20260807 04:23:41.339165 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12411
I20260807 04:23:41.343060 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:41.340137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.1.244.195:41555)
0807 04:23:41.340791 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:41.340798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:41.340829 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:23:41.340898 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:41.340904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:41.341116 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:41.341245 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:41.342312 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:41.342318 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:41.342330 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:23:41.342342 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:41.342379 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:23:41.342392 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:41.342538 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:41.342585 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 04:23:41.342702 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":331,"threads_started":1}
I20260807 04:23:41.344350 12538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:41.424613 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12411
I20260807 04:23:41.442590 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12042
I20260807 04:23:41.443828 12104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:41.547930 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12042
2026-08-07T04:23:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2343 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:23:41 dist-test-slave-zkpd krb5kdc[12565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:23:41 dist-test-slave-zkpd krb5kdc[12565](info): set up 2 sockets
Aug 07 04:23:41 dist-test-slave-zkpd krb5kdc[12565](info): commencing operation
krb5kdc: starting...
W20260807 04:23:42.667042 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.009s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:23:42 dist-test-slave-zkpd krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:23:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:23:42.927557 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41951
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:43.267163 12580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:43.267520 12580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:43.267717 12580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:43.275025 12580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:23:43.275175 12580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:43.275287 12580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:43.275399 12580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:23:43.275491 12580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:23:43.285467 12580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:43.286862 12580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:43.288164 12580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:43.299633 12585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:43.299656 12588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:43.299628 12586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:43.300452 12580 server_base.cc:1034] running on GCE node
I20260807 04:23:43.301229 12580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:43.302587 12580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:43.303815 12580 hybrid_clock.cc:648] HybridClock initialized: now 1786076623303726 us; error 75 us; skew 500 ppm
I20260807 04:23:43.304172 12580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:43 dist-test-slave-zkpd krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076623, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:43.311462 12580 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:23:43.313154 12580 webserver.cc:460] Webserver started at http://127.1.244.254:42289/ using document root <none> and password file <none>
I20260807 04:23:43.313975 12580 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:43.314144 12580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:43.314476 12580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:43.317755 12580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "8247eb2e5a864d44945816427bd5c4e5"
format_stamp: "Formatted at 2026-08-07 04:23:43 on dist-test-slave-zkpd"
server_key: "be924a8e3847118e97076f0f120ed511"
server_key_iv: "18e7690ab4cbe9e51374285fd210d506"
server_key_version: "encryptionkey@0"
I20260807 04:23:43.318595 12580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "8247eb2e5a864d44945816427bd5c4e5"
format_stamp: "Formatted at 2026-08-07 04:23:43 on dist-test-slave-zkpd"
server_key: "be924a8e3847118e97076f0f120ed511"
server_key_iv: "18e7690ab4cbe9e51374285fd210d506"
server_key_version: "encryptionkey@0"
I20260807 04:23:43.324095 12580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 04:23:43.327636 12596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:43.329005 12580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:23:43.329208 12580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "8247eb2e5a864d44945816427bd5c4e5"
format_stamp: "Formatted at 2026-08-07 04:23:43 on dist-test-slave-zkpd"
server_key: "be924a8e3847118e97076f0f120ed511"
server_key_iv: "18e7690ab4cbe9e51374285fd210d506"
server_key_version: "encryptionkey@0"
I20260807 04:23:43.329434 12580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:43.349817 12580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:43.354748 12580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:43.355069 12580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:43.375195 12580 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41951
I20260807 04:23:43.375283 12650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41951 every 8 connection(s)
I20260807 04:23:43.376647 12580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:23:43.380662 12651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:43.381456 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12580
I20260807 04:23:43.381763 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:23:43.382722 2003 external_mini_cluster.cc:1428] Setting key 94b860a4126d3ba4bd2d45253824ff3b
I20260807 04:23:43.395869 12651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5: Bootstrap starting.
Aug 07 04:23:43 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:43.399489 12651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:43.400682 12651 log.cc:824] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:43.403878 12651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5: No bootstrap required, opened a new log
I20260807 04:23:43.407490 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:43.384907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.1.244.254:41951)
0807 04:23:43.385645 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:43.385656 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:43.385697 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:23:43.387998 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:43.388021 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:43.388402 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:43.388724 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:43.390640 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:43.392002 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:43.393836 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:43.394508 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:43.394639 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:43.399272 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:43.399280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:43.399294 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:43.399329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:43.403020 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:43.403704 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:43.403712 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:43.403720 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:43.403963 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:43.404339 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:43.404346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:43.404351 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:43.405067 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:43.405381 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:43.405659 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 04:23:43.406140 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":333,"threads_started":1}
I20260807 04:23:43.413499 12651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8247eb2e5a864d44945816427bd5c4e5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41951 } }
I20260807 04:23:43.413830 12651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:43.413911 12651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8247eb2e5a864d44945816427bd5c4e5, State: Initialized, Role: FOLLOWER
I20260807 04:23:43.414359 12651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8247eb2e5a864d44945816427bd5c4e5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41951 } }
I20260807 04:23:43.414557 12651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:23:43.414651 12651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:23:43.414774 12651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:43.416929 12651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8247eb2e5a864d44945816427bd5c4e5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41951 } }
I20260807 04:23:43.417411 12651 leader_election.cc:302] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8247eb2e5a864d44945816427bd5c4e5; no voters:
I20260807 04:23:43.418270 12651 leader_election.cc:288] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:23:43.418357 12656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:43.419049 12656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [term 1 LEADER]: Becoming Leader. State: Replica: 8247eb2e5a864d44945816427bd5c4e5, State: Running, Role: LEADER
I20260807 04:23:43.419646 12656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8247eb2e5a864d44945816427bd5c4e5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41951 } }
I20260807 04:23:43.420516 12651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:23:43.423100 12657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8247eb2e5a864d44945816427bd5c4e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8247eb2e5a864d44945816427bd5c4e5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41951 } } }
I20260807 04:23:43.423614 12657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:43.423568 12658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8247eb2e5a864d44945816427bd5c4e5. Latest consensus state: current_term: 1 leader_uuid: "8247eb2e5a864d44945816427bd5c4e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8247eb2e5a864d44945816427bd5c4e5" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41951 } } }
I20260807 04:23:43.423914 12658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:23:43.424914 12663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:23:43.429533 12663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:23:43.437860 12663 catalog_manager.cc:1347] Generated new cluster ID: 7f490b6f30df470cb0038e4fadfa8f01
I20260807 04:23:43.437974 12663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:23:43.457286 12663 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:23:43.458796 12663 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:23:43.471458 12663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5: Generated new TSK 0
I20260807 04:23:43.472308 12663 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:23:43.537951 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41951
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:43.868577 12677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:43.868880 12677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:43.869053 12677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:43.876205 12677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:43.876356 12677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:43.876542 12677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:23:43.885812 12677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:43.887080 12677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:43.888461 12677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:43.901180 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:43.901250 12682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:43.901224 12685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:43.902086 12677 server_base.cc:1034] running on GCE node
I20260807 04:23:43.902704 12677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:43.903568 12677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:43.904784 12677 hybrid_clock.cc:648] HybridClock initialized: now 1786076623904731 us; error 60 us; skew 500 ppm
I20260807 04:23:43.905117 12677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:43 dist-test-slave-zkpd krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076623, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:43.908727 12677 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:23:43.909982 12677 webserver.cc:460] Webserver started at http://127.1.244.193:34963/ using document root <none> and password file <none>
I20260807 04:23:43.910557 12677 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:43.910651 12677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:43.910871 12677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:43.913658 12677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "ace7f5348a53450f8877686e9168659f"
format_stamp: "Formatted at 2026-08-07 04:23:43 on dist-test-slave-zkpd"
server_key: "d78d140a5cb598d5dbd15d85945e8184"
server_key_iv: "14090b8884cd7d279f820e528deef2fc"
server_key_version: "encryptionkey@0"
I20260807 04:23:43.914419 12677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "ace7f5348a53450f8877686e9168659f"
format_stamp: "Formatted at 2026-08-07 04:23:43 on dist-test-slave-zkpd"
server_key: "d78d140a5cb598d5dbd15d85945e8184"
server_key_iv: "14090b8884cd7d279f820e528deef2fc"
server_key_version: "encryptionkey@0"
I20260807 04:23:43.919199 12677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 04:23:43.922640 12692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:43.923831 12677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:23:43.924052 12677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "ace7f5348a53450f8877686e9168659f"
format_stamp: "Formatted at 2026-08-07 04:23:43 on dist-test-slave-zkpd"
server_key: "d78d140a5cb598d5dbd15d85945e8184"
server_key_iv: "14090b8884cd7d279f820e528deef2fc"
server_key_version: "encryptionkey@0"
I20260807 04:23:43.924218 12677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:43.949851 12677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:43.954236 12677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:43.954536 12677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:43.955490 12677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:43.956864 12677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:43.956962 12677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:43.957131 12677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:43.957196 12677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:43.989725 12677 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:42155
I20260807 04:23:43.989807 12808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:42155 every 8 connection(s)
I20260807 04:23:43.990973 12677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:23:43.993203 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12677
I20260807 04:23:43.993470 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:23:43.994309 2003 external_mini_cluster.cc:1428] Setting key fda73e20769fb2fff1fb77afbe74abae
Aug 07 04:23:44 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076623, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:44.010990 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:43.993617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:52885 (local address 127.1.244.254:41951)
0807 04:23:43.994103 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:43.994107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:43.995319 (+ 1212us) server_negotiation.cc:407] Connection header received
0807 04:23:43.997259 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:43.997264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:43.997357 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:43.997482 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:43.998851 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:43.999592 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.001140 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:44.001566 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.001707 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:44.005314 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:44.005319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:44.005322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:44.005351 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:44.007999 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:44.008711 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:44.008716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:44.008721 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:44.008846 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:44.009298 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:44.009303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:44.009306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:44.009559 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:44.009713 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:44.010383 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 04:23:44.010644 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":278,"threads_started":1}
I20260807 04:23:44.011967 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:43.994284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41951 (local address 127.1.244.193:52885)
0807 04:23:43.995112 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:43.995190 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:43.996999 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:43.997707 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:43.997715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:43.998176 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:43.998693 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:43.998710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:43.999747 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:43.999756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:44.000973 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:44.000990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:44.001833 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.001842 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:44.002070 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:44.002763 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:44.002785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:44.005108 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:44.008139 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:44.008147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:44.008162 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:44.008583 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:44.008964 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:44.008969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:44.008973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:44.009164 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:44.009688 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:44.009695 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:44.010022 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 04:23:44.010267 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 04:23:44.010787 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":313,"threads_started":1}
I20260807 04:23:44.013655 12809 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41951
I20260807 04:23:44.014014 12809 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:44.014930 12809 heartbeater.cc:511] Master 127.1.244.254:41951 requested a full tablet report, sending...
I20260807 04:23:44.017410 12614 ts_manager.cc:194] Registered new tserver with Master: ace7f5348a53450f8877686e9168659f (127.1.244.193:42155)
I20260807 04:23:44.019697 12614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:52885
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:23:44.055799 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41951
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:44.378671 12815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:44.378917 12815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:44.379084 12815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:44.386140 12815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:44.386232 12815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:44.386427 12815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:23:44.396135 12815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:44.397264 12815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:44.398604 12815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:44.411165 12820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:44.411134 12822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:44.411310 12824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:44.412186 12815 server_base.cc:1034] running on GCE node
I20260807 04:23:44.412806 12815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:44.413667 12815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:44.414948 12815 hybrid_clock.cc:648] HybridClock initialized: now 1786076624414898 us; error 52 us; skew 500 ppm
I20260807 04:23:44.415242 12815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:44 dist-test-slave-zkpd krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076624, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:44.418298 12815 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:23:44.419703 12815 webserver.cc:460] Webserver started at http://127.1.244.194:35149/ using document root <none> and password file <none>
I20260807 04:23:44.420394 12815 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:44.420537 12815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:44.420831 12815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:44.423576 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "b0147523aadb429b82eb1d4eb7f9fb6b"
format_stamp: "Formatted at 2026-08-07 04:23:44 on dist-test-slave-zkpd"
server_key: "dced28c94e29e535cfedba738e9bb2cc"
server_key_iv: "207523b99f621d4c2b284ca0202ffb79"
server_key_version: "encryptionkey@0"
I20260807 04:23:44.424425 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "b0147523aadb429b82eb1d4eb7f9fb6b"
format_stamp: "Formatted at 2026-08-07 04:23:44 on dist-test-slave-zkpd"
server_key: "dced28c94e29e535cfedba738e9bb2cc"
server_key_iv: "207523b99f621d4c2b284ca0202ffb79"
server_key_version: "encryptionkey@0"
I20260807 04:23:44.429565 12815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 04:23:44.432858 12831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:44.434175 12815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:23:44.434347 12815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "b0147523aadb429b82eb1d4eb7f9fb6b"
format_stamp: "Formatted at 2026-08-07 04:23:44 on dist-test-slave-zkpd"
server_key: "dced28c94e29e535cfedba738e9bb2cc"
server_key_iv: "207523b99f621d4c2b284ca0202ffb79"
server_key_version: "encryptionkey@0"
I20260807 04:23:44.434531 12815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:44.461705 12815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:44.465516 12815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:44.465811 12815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:44.466768 12815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:44.468132 12815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:44.468236 12815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:44.468386 12815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:44.468461 12815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:44.503378 12815 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:38187
I20260807 04:23:44.503427 12946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:38187 every 8 connection(s)
I20260807 04:23:44.504679 12815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:23:44.510073 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12815
I20260807 04:23:44.510265 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:23:44.510746 2003 external_mini_cluster.cc:1428] Setting key f6c702e36403cf1fe5c79059a4b198e6
Aug 07 04:23:44 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076624, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:44.526734 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:44.507298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41605 (local address 127.1.244.254:41951)
0807 04:23:44.507487 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:44.507492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:44.508638 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 04:23:44.510455 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:44.510461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:44.510557 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:44.510675 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:44.512554 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:44.513463 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.515996 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:44.516412 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.516516 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:44.520861 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:44.520874 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:44.520877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:44.520910 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:44.523595 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:44.524411 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:44.524416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:44.524418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:44.524542 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:44.524988 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:44.524997 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:44.524999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:44.525272 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:44.525435 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:44.526110 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 04:23:44.526363 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:23:44.527781 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:44.507774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41951 (local address 127.1.244.194:41605)
0807 04:23:44.508486 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:44.508523 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:44.510126 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:44.510937 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:44.510945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:44.511587 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:44.512374 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:44.512400 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:44.513634 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.513642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:44.515802 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:44.515821 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:44.516663 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:44.516672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:44.516990 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:44.517748 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:44.517780 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:44.520563 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:44.523745 (+ 3182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:44.523753 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:44.523768 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:44.524269 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:44.524661 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:44.524666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:44.524669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:44.524861 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:44.525402 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:44.525409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:44.525737 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 04:23:44.526019 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 04:23:44.526527 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":251,"threads_started":1}
I20260807 04:23:44.529345 12947 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41951
I20260807 04:23:44.529667 12947 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:44.530388 12947 heartbeater.cc:511] Master 127.1.244.254:41951 requested a full tablet report, sending...
I20260807 04:23:44.531975 12614 ts_manager.cc:194] Registered new tserver with Master: b0147523aadb429b82eb1d4eb7f9fb6b (127.1.244.194:38187)
I20260807 04:23:44.533289 12614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41605
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:23:44.570560 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41951
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:33599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:23:44.912070 12952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:23:44.912319 12952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:23:44.912410 12952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:23:44.919767 12952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:23:44.919857 12952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:23:44.919981 12952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:23:44.929704 12952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:23:44.930806 12952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:23:44.932065 12952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:23:44.944896 12957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:44.944849 12960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:23:44.944945 12958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:23:44.945935 12952 server_base.cc:1034] running on GCE node
I20260807 04:23:44.946601 12952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:23:44.947461 12952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:23:44.948685 12952 hybrid_clock.cc:648] HybridClock initialized: now 1786076624948640 us; error 47 us; skew 500 ppm
I20260807 04:23:44.948997 12952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:23:44 dist-test-slave-zkpd krb5kdc[12565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076624, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:23:44.952219 12952 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:23:44.953475 12952 webserver.cc:460] Webserver started at http://127.1.244.195:39891/ using document root <none> and password file <none>
I20260807 04:23:44.954114 12952 fs_manager.cc:359] Metadata directory not provided
I20260807 04:23:44.954212 12952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:23:44.954452 12952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:23:44.957247 12952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "5c9f636301844f219203710651517c89"
format_stamp: "Formatted at 2026-08-07 04:23:44 on dist-test-slave-zkpd"
server_key: "e8965dcd10e6b064c3d523b557d13f3b"
server_key_iv: "902cb3d596bfba7e7f24fcff372259ba"
server_key_version: "encryptionkey@0"
I20260807 04:23:44.957957 12952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "5c9f636301844f219203710651517c89"
format_stamp: "Formatted at 2026-08-07 04:23:44 on dist-test-slave-zkpd"
server_key: "e8965dcd10e6b064c3d523b557d13f3b"
server_key_iv: "902cb3d596bfba7e7f24fcff372259ba"
server_key_version: "encryptionkey@0"
I20260807 04:23:44.962755 12952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:23:44.965972 12967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:44.967164 12952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:23:44.967298 12952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "5c9f636301844f219203710651517c89"
format_stamp: "Formatted at 2026-08-07 04:23:44 on dist-test-slave-zkpd"
server_key: "e8965dcd10e6b064c3d523b557d13f3b"
server_key_iv: "902cb3d596bfba7e7f24fcff372259ba"
server_key_version: "encryptionkey@0"
I20260807 04:23:44.967451 12952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:23:44.986824 12952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:23:44.991384 12952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:23:44.991730 12952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:23:44.992781 12952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:23:44.994199 12952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:23:44.994304 12952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:44.994462 12952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:23:44.994534 12952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:23:45.023484 12809 heartbeater.cc:503] Master 127.1.244.254:41951 was elected leader, sending a full tablet report...
I20260807 04:23:45.028637 12952 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:38825
I20260807 04:23:45.028656 13082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:38825 every 8 connection(s)
I20260807 04:23:45.029902 12952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:23:45.035597 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 12952
I20260807 04:23:45.035812 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:23:45.036280 2003 external_mini_cluster.cc:1428] Setting key c2bc77e73acc9a4ee9ff099f7dfb1511
Aug 07 04:23:45 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076624, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:23:45.049299 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.032373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35751 (local address 127.1.244.254:41951)
0807 04:23:45.032799 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.032804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.033757 (+ 953us) server_negotiation.cc:407] Connection header received
0807 04:23:45.035588 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.035594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.035704 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.035832 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:45.037640 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.038788 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.040567 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.040952 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.041055 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.044263 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:45.044268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:45.044271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:45.044301 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:45.046663 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.047312 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.047316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.047318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.047437 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.047838 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.047841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.047843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.048166 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:45.048280 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.048875 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.049023 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":228,"threads_started":1}
I20260807 04:23:45.050274 13086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.032828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41951 (local address 127.1.244.195:35751)
0807 04:23:45.033596 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.033635 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:45.035334 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:45.036091 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:45.036099 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:45.036628 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:23:45.037402 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.037433 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.038945 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.038951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.040439 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.040451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.041134 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.041139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.041321 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.041927 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:23:45.041952 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:23:45.044098 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:23:45.046787 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:45.046794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:45.046804 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:45.047215 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:45.047547 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:23:45.047550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:23:45.047552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:23:45.047728 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:23:45.048277 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:23:45.048282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:23:45.048570 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 04:23:45.048781 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 04:23:45.049133 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":294,"threads_started":1}
I20260807 04:23:45.051661 13083 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41951
I20260807 04:23:45.052017 13083 heartbeater.cc:464] Registering TS with master...
I20260807 04:23:45.052712 13083 heartbeater.cc:511] Master 127.1.244.254:41951 requested a full tablet report, sending...
I20260807 04:23:45.054041 12614 ts_manager.cc:194] Registered new tserver with Master: 5c9f636301844f219203710651517c89 (127.1.244.195:38825)
I20260807 04:23:45.054836 12614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:35751
I20260807 04:23:45.063743 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:23:45.079510 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.068050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.1.244.254:41951)
0807 04:23:45.068252 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.068261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.068692 (+ 431us) server_negotiation.cc:407] Connection header received
0807 04:23:45.068807 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.068810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.068871 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.068947 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:45.070252 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.070903 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.072258 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.072650 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.072742 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.074910 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:45.074914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:45.074917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:45.074944 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:45.077258 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.077863 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.077866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.077869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.077983 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.078295 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.078298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.078300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.078580 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:45.078757 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.079035 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.079185 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:23:45.083033 12614 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:23:45.090077 12614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:23:45.113307 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.102975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38187 (local address 127.0.0.1:44688)
0807 04:23:45.103937 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.103955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:45.104153 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:45.105196 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:45.105203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:45.105249 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:45.105799 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.105816 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.108266 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.108271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.110534 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.110547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.112230 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.112237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.112457 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.112509 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 04:23:45.112633 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 04:23:45.112809 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":190,"spinlock_wait_cycles":23680,"thread_start_us":671,"threads_started":1}
I20260807 04:23:45.114144 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.101823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.1.244.195:38825)
0807 04:23:45.103178 (+ 1355us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.103191 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.103982 (+ 791us) server_negotiation.cc:407] Connection header received
0807 04:23:45.104439 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.104448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.104683 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.104887 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:45.106397 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.108080 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.111160 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.112644 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.112772 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.113023 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.113605 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.113791 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1161,"thread_start_us":975,"threads_started":1}
I20260807 04:23:45.114230 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.102478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.1.244.194:38187)
0807 04:23:45.103483 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.103503 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.104033 (+ 530us) server_negotiation.cc:407] Connection header received
0807 04:23:45.104440 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.104448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.104683 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.104887 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:45.106435 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.108080 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.110680 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.112033 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.112180 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.112365 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.113717 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.113907 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":702,"threads_started":1}
I20260807 04:23:45.114605 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.104483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.1.244.193:42155)
0807 04:23:45.106228 (+ 1745us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.106234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.106253 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:23:45.106321 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.106326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.106478 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.106649 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:45.107202 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.108296 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.112014 (+ 3718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.113472 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.113597 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.113842 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.114031 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.114178 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1613,"thread_start_us":1468,"threads_started":1}
I20260807 04:23:45.114509 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.104197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54732)
0807 04:23:45.105751 (+ 1554us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.105775 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:45.105882 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:45.106639 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:45.106643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:45.106667 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:45.107039 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.107048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.108475 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.108487 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.110998 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.111013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.113656 (+ 2643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.113662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.113841 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.113874 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:23:45.113978 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 04:23:45.114099 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":60,"thread_start_us":760,"threads_started":1}
I20260807 04:23:45.115072 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.101549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38825 (local address 127.0.0.1:44884)
0807 04:23:45.103792 (+ 2243us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.103837 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:45.104154 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:45.105196 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:45.105203 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:45.105253 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:45.106217 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.106225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.108213 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.108219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.110999 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.111013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.113210 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.113218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.113389 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.113424 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:23:45.114434 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0807 04:23:45.114586 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1909,"mutex_wait_us":941,"spinlock_wait_cycles":37376,"thread_start_us":1761,"threads_started":1}
I20260807 04:23:45.118678 13017 tablet_service.cc:1467] Processing CreateTablet for tablet bb5fafd922e647c99e47df1c53d96b35 (DEFAULT_TABLE table=test-table [id=89b1305ec0c34dea9f192113edd99175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:45.118670 12881 tablet_service.cc:1467] Processing CreateTablet for tablet bb5fafd922e647c99e47df1c53d96b35 (DEFAULT_TABLE table=test-table [id=89b1305ec0c34dea9f192113edd99175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:45.118670 12742 tablet_service.cc:1467] Processing CreateTablet for tablet bb5fafd922e647c99e47df1c53d96b35 (DEFAULT_TABLE table=test-table [id=89b1305ec0c34dea9f192113edd99175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:23:45.119998 13017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb5fafd922e647c99e47df1c53d96b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:45.120036 12881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb5fafd922e647c99e47df1c53d96b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:45.120056 12742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb5fafd922e647c99e47df1c53d96b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:23:45.132144 13102 tablet_bootstrap.cc:492] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Bootstrap starting.
I20260807 04:23:45.134935 13104 tablet_bootstrap.cc:492] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Bootstrap starting.
I20260807 04:23:45.136039 13103 tablet_bootstrap.cc:492] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Bootstrap starting.
I20260807 04:23:45.136406 13102 tablet_bootstrap.cc:654] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:45.137770 13102 log.cc:824] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:45.138553 13104 tablet_bootstrap.cc:654] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:45.138805 13103 tablet_bootstrap.cc:654] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Neither blocks nor log segments found. Creating new log.
I20260807 04:23:45.139850 13104 log.cc:824] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:45.140158 13103 log.cc:824] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:23:45.143747 13104 tablet_bootstrap.cc:492] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: No bootstrap required, opened a new log
I20260807 04:23:45.144088 13104 ts_tablet_manager.cc:1397] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260807 04:23:45.146337 13102 tablet_bootstrap.cc:492] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: No bootstrap required, opened a new log
I20260807 04:23:45.146533 13103 tablet_bootstrap.cc:492] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: No bootstrap required, opened a new log
I20260807 04:23:45.146654 13102 ts_tablet_manager.cc:1397] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260807 04:23:45.146791 13103 ts_tablet_manager.cc:1397] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260807 04:23:45.154137 13102 raft_consensus.cc:348] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.154510 13102 raft_consensus.cc:374] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:45.154642 13102 raft_consensus.cc:729] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9f636301844f219203710651517c89, State: Initialized, Role: FOLLOWER
I20260807 04:23:45.154724 13104 raft_consensus.cc:348] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.155167 13104 raft_consensus.cc:374] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:45.155263 13104 raft_consensus.cc:729] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0147523aadb429b82eb1d4eb7f9fb6b, State: Initialized, Role: FOLLOWER
I20260807 04:23:45.155315 13102 consensus_queue.cc:260] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.155833 13104 consensus_queue.cc:260] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.156544 13083 heartbeater.cc:503] Master 127.1.244.254:41951 was elected leader, sending a full tablet report...
I20260807 04:23:45.157009 12947 heartbeater.cc:503] Master 127.1.244.254:41951 was elected leader, sending a full tablet report...
I20260807 04:23:45.157248 13102 ts_tablet_manager.cc:1428] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260807 04:23:45.157665 13104 ts_tablet_manager.cc:1428] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 04:23:45.158533 13103 raft_consensus.cc:348] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.158828 13103 raft_consensus.cc:374] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:23:45.158951 13103 raft_consensus.cc:729] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace7f5348a53450f8877686e9168659f, State: Initialized, Role: FOLLOWER
I20260807 04:23:45.159444 13103 consensus_queue.cc:260] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.160976 13103 ts_tablet_manager.cc:1428] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260807 04:23:45.243366 12810 tablet.cc:2369] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:23:45.256770 12948 tablet.cc:2369] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:23:45.281195 13110 raft_consensus.cc:482] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 04:23:45.281586 13084 tablet.cc:2369] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:23:45.281611 13110 raft_consensus.cc:504] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.283553 13110 leader_election.cc:288] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c9f636301844f219203710651517c89 (127.1.244.195:38825), b0147523aadb429b82eb1d4eb7f9fb6b (127.1.244.194:38187)
I20260807 04:23:45.290661 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.284226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:46867 (local address 127.1.244.194:38187)
0807 04:23:45.284452 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.284458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.284945 (+ 487us) server_negotiation.cc:407] Connection header received
0807 04:23:45.285146 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.285150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.285208 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.285304 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:45.285934 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.286606 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.288532 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.289549 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.289703 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.289792 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.290268 (+ 476us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.290411 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 04:23:45.290661 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.283808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41389 (local address 127.1.244.195:38825)
0807 04:23:45.284100 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.284110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.284765 (+ 655us) server_negotiation.cc:407] Connection header received
0807 04:23:45.284999 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.285003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.285061 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.285160 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:23:45.285804 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.286536 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.288507 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.289607 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.289788 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.289862 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.290294 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.290414 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:23:45.290938 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.283849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:38187 (local address 127.1.244.193:46867)
0807 04:23:45.284840 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.284855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:45.285008 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:45.285325 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:45.285329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:45.285368 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:45.285758 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.285767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.286747 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.286752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.288381 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.288394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.289835 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.289842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.290066 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.290120 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 04:23:45.290212 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 04:23:45.290471 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":113,"thread_start_us":618,"threads_started":1}
I20260807 04:23:45.290970 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.283762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38825 (local address 127.1.244.193:41389)
0807 04:23:45.284624 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.284654 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 04:23:45.284846 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:23:45.285175 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:23:45.285180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:23:45.285203 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:23:45.285642 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.285655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.286663 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.286671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.288357 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:23:45.288369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.289955 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.289960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:23:45.290128 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.290153 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:23:45.290220 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 04:23:45.290471 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":143,"spinlock_wait_cycles":16640,"thread_start_us":701,"threads_started":1}
I20260807 04:23:45.291966 12901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb5fafd922e647c99e47df1c53d96b35" candidate_uuid: "ace7f5348a53450f8877686e9168659f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" is_pre_election: true
I20260807 04:23:45.292007 13037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb5fafd922e647c99e47df1c53d96b35" candidate_uuid: "ace7f5348a53450f8877686e9168659f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9f636301844f219203710651517c89" is_pre_election: true
I20260807 04:23:45.292460 13037 raft_consensus.cc:2454] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace7f5348a53450f8877686e9168659f in term 0.
I20260807 04:23:45.292443 12901 raft_consensus.cc:2454] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ace7f5348a53450f8877686e9168659f in term 0.
I20260807 04:23:45.293076 12695 leader_election.cc:302] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5c9f636301844f219203710651517c89, ace7f5348a53450f8877686e9168659f, b0147523aadb429b82eb1d4eb7f9fb6b; no voters:
I20260807 04:23:45.293447 13110 raft_consensus.cc:2781] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:23:45.293587 13110 raft_consensus.cc:482] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:23:45.293695 13110 raft_consensus.cc:3037] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:45.295866 13110 raft_consensus.cc:504] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.296614 13110 leader_election.cc:288] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [CANDIDATE]: Term 1 election: Requested vote from peers 5c9f636301844f219203710651517c89 (127.1.244.195:38825), b0147523aadb429b82eb1d4eb7f9fb6b (127.1.244.194:38187)
I20260807 04:23:45.296947 13037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb5fafd922e647c99e47df1c53d96b35" candidate_uuid: "ace7f5348a53450f8877686e9168659f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9f636301844f219203710651517c89"
I20260807 04:23:45.297070 12901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb5fafd922e647c99e47df1c53d96b35" candidate_uuid: "ace7f5348a53450f8877686e9168659f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b"
I20260807 04:23:45.297142 13037 raft_consensus.cc:3037] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:45.297295 12901 raft_consensus.cc:3037] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:23:45.299281 13037 raft_consensus.cc:2454] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace7f5348a53450f8877686e9168659f in term 1.
I20260807 04:23:45.299678 12694 leader_election.cc:302] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c9f636301844f219203710651517c89, ace7f5348a53450f8877686e9168659f; no voters:
I20260807 04:23:45.299965 12901 raft_consensus.cc:2454] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ace7f5348a53450f8877686e9168659f in term 1.
I20260807 04:23:45.299993 13110 raft_consensus.cc:2781] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:23:45.300741 13110 raft_consensus.cc:686] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 1 LEADER]: Becoming Leader. State: Replica: ace7f5348a53450f8877686e9168659f, State: Running, Role: LEADER
I20260807 04:23:45.301236 13110 consensus_queue.cc:237] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } }
I20260807 04:23:45.305420 12614 catalog_manager.cc:5495] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f reported cstate change: term changed from 0 to 1, leader changed from <none> to ace7f5348a53450f8877686e9168659f (127.1.244.193). New cstate: current_term: 1 leader_uuid: "ace7f5348a53450f8877686e9168659f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ace7f5348a53450f8877686e9168659f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 } health_report { overall_health: UNKNOWN } } }
I20260807 04:23:45.346823 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.342014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.1.244.193:42155)
0807 04:23:45.342187 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.342191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.342206 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:23:45.342284 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.342288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.342362 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.342480 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:23:45.342818 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.343509 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.344949 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.345319 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.345409 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.345827 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:23:45.346278 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:23:45.346379 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.346478 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.346536 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:23:45.354781 13037 raft_consensus.cc:1261] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 1 FOLLOWER]: Refusing update from remote peer ace7f5348a53450f8877686e9168659f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:45.354781 12901 raft_consensus.cc:1261] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 1 FOLLOWER]: Refusing update from remote peer ace7f5348a53450f8877686e9168659f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:23:45.355871 13113 consensus_queue.cc:1035] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0147523aadb429b82eb1d4eb7f9fb6b" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 38187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:45.356263 13110 consensus_queue.cc:1035] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c9f636301844f219203710651517c89" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:23:45.367501 13116 mvcc.cc:205] Tried to move back new op lower bound from 7315769857442471936 to 7315769857237479424. Current Snapshot: MvccSnapshot[applied={T|T < 7315769857442471936}]
I20260807 04:23:45.369660 13118 mvcc.cc:205] Tried to move back new op lower bound from 7315769857442471936 to 7315769857237479424. Current Snapshot: MvccSnapshot[applied={T|T < 7315769857442471936}]
I20260807 04:23:45.381767 12612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:45.382133 12612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:23:45.386343 12612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8247eb2e5a864d44945816427bd5c4e5: Sending DeleteTablet for 3 replicas of tablet bb5fafd922e647c99e47df1c53d96b35
I20260807 04:23:45.387522 12881 tablet_service.cc:1514] Processing DeleteTablet for tablet bb5fafd922e647c99e47df1c53d96b35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:45 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44688
I20260807 04:23:45.387548 13017 tablet_service.cc:1514] Processing DeleteTablet for tablet bb5fafd922e647c99e47df1c53d96b35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:45 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:44884
I20260807 04:23:45.387738 12742 tablet_service.cc:1514] Processing DeleteTablet for tablet bb5fafd922e647c99e47df1c53d96b35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:23:45 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:54732
I20260807 04:23:45.393613 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12677
I20260807 04:23:45.402079 13125 tablet_replica.cc:330] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: stopping tablet replica
I20260807 04:23:45.402623 13125 raft_consensus.cc:2229] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:45.403226 13125 raft_consensus.cc:2258] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:45.405558 13125 ts_tablet_manager.cc:1905] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:45.407913 13126 tablet_replica.cc:330] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: stopping tablet replica
I20260807 04:23:45.408525 13126 raft_consensus.cc:2229] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:23:45.409091 13126 raft_consensus.cc:2258] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:45.412966 13125 ts_tablet_manager.cc:1918] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:45.413169 13125 log.cc:1196] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/bb5fafd922e647c99e47df1c53d96b35
I20260807 04:23:45.413378 13126 ts_tablet_manager.cc:1905] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:45.413779 13125 ts_tablet_manager.cc:1939] T bb5fafd922e647c99e47df1c53d96b35 P b0147523aadb429b82eb1d4eb7f9fb6b: Deleting consensus metadata
I20260807 04:23:45.415545 12599 catalog_manager.cc:4841] TS b0147523aadb429b82eb1d4eb7f9fb6b (127.1.244.194:38187): tablet bb5fafd922e647c99e47df1c53d96b35 (table test-table [id=89b1305ec0c34dea9f192113edd99175]) successfully deleted
Aug 07 04:23:45 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:23:45.420490 13127 tablet_replica.cc:330] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: stopping tablet replica
I20260807 04:23:45.421011 13127 raft_consensus.cc:2229] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:23:45.421532 13126 ts_tablet_manager.cc:1918] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:23:45.421754 13126 log.cc:1196] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/bb5fafd922e647c99e47df1c53d96b35
I20260807 04:23:45.421787 13127 raft_consensus.cc:2258] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:23:45.422385 13126 ts_tablet_manager.cc:1939] T bb5fafd922e647c99e47df1c53d96b35 P 5c9f636301844f219203710651517c89: Deleting consensus metadata
I20260807 04:23:45.424309 12598 catalog_manager.cc:4841] TS 5c9f636301844f219203710651517c89 (127.1.244.195:38825): tablet bb5fafd922e647c99e47df1c53d96b35 (table test-table [id=89b1305ec0c34dea9f192113edd99175]) successfully deleted
I20260807 04:23:45.424453 13127 ts_tablet_manager.cc:1905] T bb5fafd922e647c99e47df1c53d96b35 P ace7f5348a53450f8877686e9168659f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:23:45.426493 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.394696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54746 (local address 127.1.244.193:42155)
0807 04:23:45.394858 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.394865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.401432 (+ 6567us) server_negotiation.cc:407] Connection header received
0807 04:23:45.401622 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.401631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.401729 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.401842 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:45.403967 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.405142 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.406937 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.407633 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.407724 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.417991 (+ 10267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:45.418023 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:45.418039 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:45.418091 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:45.422584 (+ 4493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.423680 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.423693 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.423696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.423884 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.424401 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.424406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.424409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.424859 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:45.425055 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.425621 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.425891 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:23:45.427474 12804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:45.530843 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12677
I20260807 04:23:45.551288 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12815
W20260807 04:23:45.551229 12598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:42155 (error 108)
I20260807 04:23:45.553236 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.552821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54758)
0807 04:23:45.553000 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.553101 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 04:23:45.553623 12598 catalog_manager.cc:4584] TS ace7f5348a53450f8877686e9168659f (127.1.244.193:42155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb5fafd922e647c99e47df1c53d96b35: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Aug 07 04:23:45 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:23:45.565910 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.552170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.1.244.194:38187)
0807 04:23:45.552352 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.552358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.552381 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:23:45.552609 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.552613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.552677 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.552759 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:45.554294 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.555034 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.556318 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.556708 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.556800 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.560744 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:45.560755 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:45.560768 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:45.560797 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:45.563360 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.564253 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.564260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.564262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.564372 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.564706 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.564709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.564711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.565081 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:45.565287 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.565525 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.565668 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:23:45.566497 12942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:45.607354 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.606803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54766)
0807 04:23:45.607028 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.607133 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 04:23:45.608454 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.608069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54778)
0807 04:23:45.608219 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.608324 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 04:23:45.654970 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12815
I20260807 04:23:45.669121 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.668514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54780)
0807 04:23:45.668785 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.668903 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 04:23:45.670255 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.669815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54788)
0807 04:23:45.670011 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.670101 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 04:23:45.673930 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12952
Aug 07 04:23:45 dist-test-slave-zkpd krb5kdc[12565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:23:45.688107 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.674647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.1.244.195:38825)
0807 04:23:45.674816 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:23:45.674821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:23:45.674838 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:23:45.674982 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:23:45.674986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:23:45.675049 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:23:45.675128 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:23:45.676454 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.677154 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.678241 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:23:45.678590 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:23:45.678683 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:23:45.682901 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:23:45.682916 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:23:45.682932 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:23:45.682961 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:23:45.685540 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.686239 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.686245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.686248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.686370 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:23:45.686666 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:23:45.686669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:23:45.686671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:23:45.686969 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:23:45.687171 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 04:23:45.687475 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 04:23:45.687616 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:23:45.688789 13078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:45.750900 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.750338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54800)
0807 04:23:45.750591 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.750698 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 04:23:45.752111 13099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:23:45.751665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42155 (local address 127.0.0.1:54812)
0807 04:23:45.751875 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 04:23:45.751987 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:42155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 04:23:45.778211 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12952
I20260807 04:23:45.797314 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 12580
I20260807 04:23:45.798499 12646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:23:45.906682 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 12580
2026-08-07T04:23:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4360 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:23:46 dist-test-slave-zkpd krb5kdc[13136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:23:46 dist-test-slave-zkpd krb5kdc[13136](info): set up 2 sockets
Aug 07 04:23:46 dist-test-slave-zkpd krb5kdc[13136](info): commencing operation
krb5kdc: starting...
W20260807 04:23:47.059803 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:23:47 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:23:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
I20260807 04:23:47.422595 2003 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskA8s8yC/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/postgres -l logfile start
2026-08-07 04:23:49.748 UTC [13168] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 04:23:49.748 UTC [13168] LOG: listening on IPv4 address "127.1.244.212", port 48145
2026-08-07 04:23:49.751 UTC [13168] LOG: listening on Unix socket "/tmp/.s.PGSQL.48145"
2026-08-07 04:23:49.773 UTC [13171] LOG: database system was shut down at 2026-08-07 04:23:48 UTC
2026-08-07 04:23:49.776 UTC [13168] LOG: database system is ready to accept connections
I20260807 04:23:50.759404 2003 mini_postgres.cc:96] Postgres bound to 48145
2026-08-07 04:23:50.782 UTC [13179] FATAL: database "slave" does not exist
127.1.244.212:48145 - accepting connections
I20260807 04:23:50.782949 2003 mini_ranger.cc:161] Starting Ranger...
I20260807 04:23:50.810376 2003 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 04:23:51.220964 2003 mini_ranger.cc:87] Created ranger Postgres database
I20260807 04:23:51.221109 2003 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-admin
2026-08-07 04:23:51,520 [I] DB FLAVOR :POSTGRES
2026-08-07 04:23:51,521 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:23:51,521 [I] Checking connection..
2026-08-07 04:23:51,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 04:23:51,779 [I] Checking connection passed.
2026-08-07 04:23:51,780 [I] --------- Verifying version history table ---------
2026-08-07 04:23:51,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:23:52,040 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:23:52,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:23:52,308 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:23:52,308 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 04:23:52,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 04:23:52.548 UTC [13246] WARNING: there is no transaction in progress
2026-08-07 04:23:52,554 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 04:23:52,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:23:52,794 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 04:23:52,795 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:23:52,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 04:23:53,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 04:23:53,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-07 04:23:53,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:23:53,788 [I] Table x_portal_user does not exist in database ranger
2026-08-07 04:23:53,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:23:54,042 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 04:23:54,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:23:54,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:23:54,531 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 04:23:54,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 04:23:55.391 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.406 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.421 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.435 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.449 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.468 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.518 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.524 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.536 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.546 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.555 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.569 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.578 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.584 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.589 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:55.595 UTC [13357] WARNING: there is no transaction in progress
2026-08-07 04:23:56,144 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 04:23:56,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 04:23:56,394 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 04:23:56,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:23:56,653 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 04:23:56,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:23:56,878 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 04:23:56,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:23:57,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:23:57,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-07 04:23:57,578 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 04:23:57,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 04:23:57,819 [I] DB_PATCHES have already been applied
I20260807 04:23:57.825140 2003 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-admin:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskA8s8yC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 04:23:57.825227 2003 mini_ranger.cc:193] Using host: 127.1.244.212
I20260807 04:23:57.845543 2003 mini_ranger.cc:227] Ranger admin URL: http://127.1.244.212:47145
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 4:23:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 04:23:58.986188 2003 mini_ranger.cc:160] Time spent starting Ranger: real 8.203s user 0.000s sys 0.053s
Aug 07, 2026 4:24:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 4:24:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 4:24:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 4:24:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 4:24:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 04:24:19.542129 2003 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.244.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 04:24:19 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076659, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 04:24:19.752254 2003 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 04:24:19.777979 2003 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 04:24:20.553555 2003 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 04:24:20.553750 2003 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms
I20260807 04:24:20.553879 2003 mini_ranger_kms.cc:227] Using postgres at 127.1.244.212:48145
2026-08-07 04:24:21,044 [I] DB FLAVOR :POSTGRES
2026-08-07 04:24:21,044 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:24:21,044 [I] Checking connection
2026-08-07 04:24:21,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 04:24:21,285 [I] connection success
2026-08-07 04:24:21,285 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 04:24:21,285 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 04:24:21,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 04:24:21,514 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 04:24:21,515 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:24:21,515 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 04:24:21,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:48145/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 04:24:21,783 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 04:24:22.220155 2003 mini_ranger_kms.cc:310] Created kms service
I20260807 04:24:22.388303 2003 mini_ranger_kms.cc:326] Created kudu user
I20260807 04:24:22.429361 2003 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 04:24:22 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076659, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:24:22.826817 2003 mini_ranger_kms.cc:384] Added ranger policy
I20260807 04:24:22.828218 2003 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms:/tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 04:24:22.828328 2003 mini_ranger_kms.cc:240] Using host: 127.1.244.212
I20260807 04:24:22.857385 2003 mini_ranger_kms.cc:276] Ranger KMS PID: 13544
I20260807 04:24:22.857566 2003 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.244.212:49815
Aug 07, 2026 4:24:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:24:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:24:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 4:24:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 4:24:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 4:24:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 04:24:25 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786076665, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:24:25.221593 2003 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.469s user 0.113s sys 0.408s
I20260807 04:24:25.221984 2003 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 04:24:25.222203 2003 mini_ranger_kms.cc:401] 127.1.244.212:49815/kms/v1/keys
Aug 07 04:24:25 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076665, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Aug 07 04:24:25 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076665, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Aug 07 04:24:28 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:24:28.348918 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37519
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:37519
--ranger_config_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:49815/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:24:28.713605 13596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:24:28.713912 13596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:24:28.714074 13596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:24:28.721757 13596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:24:28.721874 13596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:24:28.721951 13596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:24:28.722047 13596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:24:28.722119 13596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:24:28.732016 13596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.244.212:49815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:37519
--ranger_config_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:37519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:24:28.733249 13596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:24:28.734566 13596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:24:28.747187 13601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:28.747287 13604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:28.747225 13602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:24:28.748637 13596 server_base.cc:1034] running on GCE node
I20260807 04:24:28.749399 13596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:24:28.750799 13596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:24:28.751995 13596 hybrid_clock.cc:648] HybridClock initialized: now 1786076668751941 us; error 42 us; skew 500 ppm
I20260807 04:24:28.752331 13596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:24:28 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:24:28.755501 13596 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:24:28.756883 13596 webserver.cc:460] Webserver started at http://127.1.244.254:41043/ using document root <none> and password file <none>
I20260807 04:24:28.757529 13596 fs_manager.cc:359] Metadata directory not provided
I20260807 04:24:28.757663 13596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:24:28.757965 13596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:24:28 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:24:28.821795 13596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "88422a3d0a3249a7a33de5ff198f79c9"
format_stamp: "Formatted at 2026-08-07 04:24:28 on dist-test-slave-zkpd"
server_key: "dd3c57505796c54b915f3e71caccec0e"
server_key_iv: "80c1289fab1583aaa84c02b3b7d3e947"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:28.822861 13596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "88422a3d0a3249a7a33de5ff198f79c9"
format_stamp: "Formatted at 2026-08-07 04:24:28 on dist-test-slave-zkpd"
server_key: "dd3c57505796c54b915f3e71caccec0e"
server_key_iv: "80c1289fab1583aaa84c02b3b7d3e947"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:28.829872 13596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 04:24:28.846900 13611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:28.848543 13596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 04:24:28.848757 13596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "88422a3d0a3249a7a33de5ff198f79c9"
format_stamp: "Formatted at 2026-08-07 04:24:28 on dist-test-slave-zkpd"
server_key: "dd3c57505796c54b915f3e71caccec0e"
server_key_iv: "80c1289fab1583aaa84c02b3b7d3e947"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:28.848995 13596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:24:28.868381 13596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:24:28.871877 13596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:24:28.872203 13596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:24:28.891445 13596 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:37519
I20260807 04:24:28.891479 13663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:37519 every 8 connection(s)
I20260807 04:24:28.892838 13596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:24:28.894618 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 13596
I20260807 04:24:28.894840 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:24:28.895720 2003 external_mini_cluster.cc:1428] Setting key f7167d7a7dbcef61bb75145be0e6c624
I20260807 04:24:28.897812 13664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:24:28 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:24:28.911307 13664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9: Bootstrap starting.
I20260807 04:24:28.914518 13664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9: Neither blocks nor log segments found. Creating new log.
I20260807 04:24:28.915638 13664 log.cc:824] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:24:28.918124 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:28.898016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.1.244.254:37519)
0807 04:24:28.898752 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:28.898763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:28.898805 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:24:28.900473 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:28.900493 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:28.900859 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:28.901243 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:28.902742 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:28.904140 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:28.905884 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:28.906533 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:28.906655 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:28.911089 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:28.911096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:28.911109 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:28.911143 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:28.914153 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:28.914867 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:28.914874 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:28.914879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:28.915043 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:28.915435 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:28.915441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:28.915448 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:28.916088 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:28.916409 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:28.916835 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 04:24:28.917216 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":354,"threads_started":1}
I20260807 04:24:28.918628 13664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9: No bootstrap required, opened a new log
I20260807 04:24:28.926601 13664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88422a3d0a3249a7a33de5ff198f79c9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37519 } }
I20260807 04:24:28.926975 13664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:24:28.927104 13664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88422a3d0a3249a7a33de5ff198f79c9, State: Initialized, Role: FOLLOWER
I20260807 04:24:28.927657 13664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88422a3d0a3249a7a33de5ff198f79c9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37519 } }
I20260807 04:24:28.927945 13664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:24:28.928129 13664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:24:28.928314 13664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:24:28.930531 13664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88422a3d0a3249a7a33de5ff198f79c9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37519 } }
I20260807 04:24:28.931102 13664 leader_election.cc:302] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88422a3d0a3249a7a33de5ff198f79c9; no voters:
I20260807 04:24:28.931751 13664 leader_election.cc:288] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:24:28.931869 13669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:24:28.932721 13669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [term 1 LEADER]: Becoming Leader. State: Replica: 88422a3d0a3249a7a33de5ff198f79c9, State: Running, Role: LEADER
I20260807 04:24:28.933279 13669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88422a3d0a3249a7a33de5ff198f79c9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37519 } }
I20260807 04:24:28.933826 13664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:24:28.936269 13672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88422a3d0a3249a7a33de5ff198f79c9. Latest consensus state: current_term: 1 leader_uuid: "88422a3d0a3249a7a33de5ff198f79c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88422a3d0a3249a7a33de5ff198f79c9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37519 } } }
I20260807 04:24:28.936326 13670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88422a3d0a3249a7a33de5ff198f79c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88422a3d0a3249a7a33de5ff198f79c9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 37519 } } }
I20260807 04:24:28.936683 13672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:24:28.936683 13670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:24:28.938802 13678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:24:28.944550 13678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:24:28.945147 13664 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 04:24:28.961212 13678 catalog_manager.cc:1347] Generated new cluster ID: ce313a2fc19549bf95d189aa4e9e85d2
I20260807 04:24:28.961382 13678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:24:28.975813 13678 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:24:28.977967 13678 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:24:28.988798 13678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9: Generated new TSK 0
I20260807 04:24:28.989821 13678 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:24:29.740532 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:49815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37519
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:24:30 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786076670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 04:24:30.183194 13711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:24:30.183686 13711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:24:30.183933 13711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:24:30.197052 13711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:24:30.197260 13711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:24:30.197512 13711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:24:30.215394 13711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.244.212:49815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:24:30.217406 13711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:24:30.219336 13711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:24:30.241629 13721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:30.242867 13722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:30.244951 13724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:24:30.245440 13711 server_base.cc:1034] running on GCE node
I20260807 04:24:30.246176 13711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:24:30.247324 13711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:24:30.248569 13711 hybrid_clock.cc:648] HybridClock initialized: now 1786076670248491 us; error 61 us; skew 500 ppm
I20260807 04:24:30.248960 13711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:24:30 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:24:30.257197 13711 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:24:30.258925 13711 webserver.cc:460] Webserver started at http://127.1.244.193:38355/ using document root <none> and password file <none>
I20260807 04:24:30.259676 13711 fs_manager.cc:359] Metadata directory not provided
I20260807 04:24:30.259843 13711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:24:30.260218 13711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:24:30 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:24:30.279942 13711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "3c1becafa09040649cbe42eacf54f315"
format_stamp: "Formatted at 2026-08-07 04:24:30 on dist-test-slave-zkpd"
server_key: "d5a7a7d02797e9e90f316790f0d7b443"
server_key_iv: "c0d27febca8a99a1643566c5ca1f2659"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:30.281410 13711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "3c1becafa09040649cbe42eacf54f315"
format_stamp: "Formatted at 2026-08-07 04:24:30 on dist-test-slave-zkpd"
server_key: "d5a7a7d02797e9e90f316790f0d7b443"
server_key_iv: "c0d27febca8a99a1643566c5ca1f2659"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:30.306825 13711 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.003s sys 0.020s
I20260807 04:24:30.339876 13731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:30.342826 13711 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.006s
I20260807 04:24:30.343087 13711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "3c1becafa09040649cbe42eacf54f315"
format_stamp: "Formatted at 2026-08-07 04:24:30 on dist-test-slave-zkpd"
server_key: "d5a7a7d02797e9e90f316790f0d7b443"
server_key_iv: "c0d27febca8a99a1643566c5ca1f2659"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:30.343516 13711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:24:30.376425 13711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:24:30.382192 13711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:24:30.382602 13711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:24:30.390048 13711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:24:30.392463 13711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:24:30.392584 13711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:30.392823 13711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:24:30.392931 13711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 04:24:30 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:24:30.513525 13711 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:35173
I20260807 04:24:30.513550 13848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:35173 every 8 connection(s)
I20260807 04:24:30.515509 13711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:24:30.519809 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 13711
I20260807 04:24:30.520107 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:24:30.521309 2003 external_mini_cluster.cc:1428] Setting key ff8d8dfa0dbdc3c3251b4dbadafd9e69
Aug 07 04:24:30 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:24:30.573053 13852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:30.522472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51735 (local address 127.1.244.254:37519)
0807 04:24:30.548436 (+ 25964us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:30.548443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:30.552460 (+ 4017us) server_negotiation.cc:407] Connection header received
0807 04:24:30.552535 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:30.552540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:30.552649 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:30.552742 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:30.556001 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:30.557204 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:30.559837 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:30.560583 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:30.560690 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:30.565658 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:30.565665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:30.565670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:30.565718 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:30.569458 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:30.570318 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:30.570324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:30.570327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:30.570475 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:30.571006 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:30.571012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:30.571016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:30.571363 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:30.571495 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:30.572468 (+ 973us) server_negotiation.cc:299] Negotiation successful
0807 04:24:30.572681 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25799,"thread_start_us":25973,"threads_started":1}
I20260807 04:24:30.575420 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:30.521750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37519 (local address 127.1.244.193:51735)
0807 04:24:30.549194 (+ 27444us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:30.549253 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:30.552001 (+ 2748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:30.554077 (+ 2076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:30.554090 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:30.554795 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:24:30.555729 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:30.555756 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:30.557436 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:30.557442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:30.559637 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:30.559656 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:30.561053 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:30.561063 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:30.561385 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:30.562305 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:24:30.562339 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:24:30.565414 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:24:30.569637 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:24:30.569645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:24:30.569660 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:24:30.570188 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:24:30.570606 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:24:30.570611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:24:30.570615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:24:30.570859 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:24:30.571592 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:24:30.571599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:24:30.572031 (+ 432us) client_negotiation.cc:769] Sending connection context
0807 04:24:30.573582 (+ 1551us) client_negotiation.cc:240] Negotiation successful
0807 04:24:30.574100 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27062,"thread_start_us":31026,"threads_started":1}
I20260807 04:24:30.577479 13849 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37519
I20260807 04:24:30.577816 13849 heartbeater.cc:464] Registering TS with master...
I20260807 04:24:30.578498 13849 heartbeater.cc:511] Master 127.1.244.254:37519 requested a full tablet report, sending...
I20260807 04:24:30.580802 13628 ts_manager.cc:194] Registered new tserver with Master: 3c1becafa09040649cbe42eacf54f315 (127.1.244.193:35173)
I20260807 04:24:30.582911 13628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:51735
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:24:30.626576 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:49815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37519
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:24:30 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076670, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
W20260807 04:24:31.140013 13855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:24:31.140369 13855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:24:31.140542 13855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:24:31.148905 13855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:24:31.149044 13855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:24:31.149228 13855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:24:31.159508 13855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.244.212:49815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:24:31.160930 13855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:24:31.162248 13855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:24:31.187664 13862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:31.188942 13864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:31.189941 13868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:24:31.191619 13855 server_base.cc:1034] running on GCE node
I20260807 04:24:31.192600 13855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:24:31.193910 13855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:24:31.195178 13855 hybrid_clock.cc:648] HybridClock initialized: now 1786076671195120 us; error 65 us; skew 500 ppm
I20260807 04:24:31.195547 13855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:24:31 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:24:31.199519 13855 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:24:31.202198 13855 webserver.cc:460] Webserver started at http://127.1.244.194:45697/ using document root <none> and password file <none>
I20260807 04:24:31.202912 13855 fs_manager.cc:359] Metadata directory not provided
I20260807 04:24:31.203069 13855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:24:31.203394 13855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:24:31 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:24:31.220253 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "c0c193cecb2747edaa4b8d6004c5d10a"
format_stamp: "Formatted at 2026-08-07 04:24:31 on dist-test-slave-zkpd"
server_key: "8ab9655bcfe6dd8e3f28e13a02f8ea1d"
server_key_iv: "c9657f37766137e45949095a8db191a7"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:31.221475 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "c0c193cecb2747edaa4b8d6004c5d10a"
format_stamp: "Formatted at 2026-08-07 04:24:31 on dist-test-slave-zkpd"
server_key: "8ab9655bcfe6dd8e3f28e13a02f8ea1d"
server_key_iv: "c9657f37766137e45949095a8db191a7"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:31.228668 13855 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 04:24:31.246882 13880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:31.248735 13855 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:24:31.248909 13855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "c0c193cecb2747edaa4b8d6004c5d10a"
format_stamp: "Formatted at 2026-08-07 04:24:31 on dist-test-slave-zkpd"
server_key: "8ab9655bcfe6dd8e3f28e13a02f8ea1d"
server_key_iv: "c9657f37766137e45949095a8db191a7"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:31.249162 13855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:24:31.276635 13855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:24:31.281775 13855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:24:31.282191 13855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:24:31.283408 13855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:24:31.285211 13855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:24:31.285305 13855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:31.285477 13855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:24:31.285560 13855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:31.330633 13855 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:36639
I20260807 04:24:31.330675 14000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:36639 every 8 connection(s)
I20260807 04:24:31.332541 13855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:24:31.333443 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 13855
I20260807 04:24:31.333611 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:24:31.334154 2003 external_mini_cluster.cc:1428] Setting key a0934f71e5ccf7a41502cb1028d2c037
Aug 07 04:24:31 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
I20260807 04:24:31.372685 14004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:31.339087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:46579 (local address 127.1.244.254:37519)
0807 04:24:31.348059 (+ 8972us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:31.348065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:31.348091 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:24:31.353615 (+ 5524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:31.353621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:31.353728 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:31.353902 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:31.356039 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.357316 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.359945 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.360628 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.360723 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:31.365241 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:31.365248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:31.365253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:31.365303 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:31.368903 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:31.369862 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:31.369867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:31.369871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:31.370015 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:31.370569 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:31.370574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:31.370577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:31.370916 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:31.371045 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:31.372034 (+ 989us) server_negotiation.cc:299] Negotiation successful
0807 04:24:31.372289 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8807,"thread_start_us":8681,"threads_started":1}
I20260807 04:24:31.375617 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:31.338417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37519 (local address 127.1.244.194:46579)
0807 04:24:31.344642 (+ 6225us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:31.344721 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:31.347499 (+ 2778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:31.354342 (+ 6843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:31.354354 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:31.354978 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:24:31.355803 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:31.355828 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.357608 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.357613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:31.359758 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:31.359774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.361046 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.361052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:31.361368 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:31.362118 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:24:31.362147 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:24:31.364979 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:24:31.369081 (+ 4102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:24:31.369090 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:24:31.369107 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:24:31.369723 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:24:31.370160 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:24:31.370165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:24:31.370169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:24:31.370437 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:24:31.371136 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:24:31.371144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:24:31.371580 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 04:24:31.372978 (+ 1398us) client_negotiation.cc:240] Negotiation successful
0807 04:24:31.373533 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5828,"thread_start_us":5523,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
I20260807 04:24:31.377178 14001 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37519
I20260807 04:24:31.377517 14001 heartbeater.cc:464] Registering TS with master...
I20260807 04:24:31.378361 14001 heartbeater.cc:511] Master 127.1.244.254:37519 requested a full tablet report, sending...
I20260807 04:24:31.380064 13628 ts_manager.cc:194] Registered new tserver with Master: c0c193cecb2747edaa4b8d6004c5d10a (127.1.244.194:36639)
I20260807 04:24:31.381152 13628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:46579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:24:31.414942 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:49815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37519
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:24:31.586752 13849 heartbeater.cc:503] Master 127.1.244.254:37519 was elected leader, sending a full tablet report...
W20260807 04:24:31.826218 14007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:24:31.826475 14007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:24:31.826656 14007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:24:31.833998 14007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:24:31.834159 14007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:24:31.834350 14007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:24:31.844208 14007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.244.212:49815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:37519
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:24:31.845367 14007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:24:31.846614 14007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:24:31.860037 14016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:31.860126 14013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:24:31.860102 14014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:24:31.861001 14007 server_base.cc:1034] running on GCE node
I20260807 04:24:31.861500 14007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:24:31.862345 14007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:24:31.863546 14007 hybrid_clock.cc:648] HybridClock initialized: now 1786076671863506 us; error 28 us; skew 500 ppm
I20260807 04:24:31.863883 14007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:24:31 dist-test-slave-zkpd krb5kdc[13136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:24:31.867076 14007 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:24:31.868573 14007 webserver.cc:460] Webserver started at http://127.1.244.195:44443/ using document root <none> and password file <none>
I20260807 04:24:31.869210 14007 fs_manager.cc:359] Metadata directory not provided
I20260807 04:24:31.869357 14007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:24:31.869669 14007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:24:31 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:24:31.883656 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "1b8cd37af6924bfeb7770f87fea43b4e"
format_stamp: "Formatted at 2026-08-07 04:24:31 on dist-test-slave-zkpd"
server_key: "10034a80355c12d7650a513468e98ca5"
server_key_iv: "d58012f1c7bdcfd15d2cc95002bb98a8"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:31.884608 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "1b8cd37af6924bfeb7770f87fea43b4e"
format_stamp: "Formatted at 2026-08-07 04:24:31 on dist-test-slave-zkpd"
server_key: "10034a80355c12d7650a513468e98ca5"
server_key_iv: "d58012f1c7bdcfd15d2cc95002bb98a8"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:31.890004 14007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260807 04:24:31.907233 14023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:31.908846 14007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:24:31.909046 14007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "1b8cd37af6924bfeb7770f87fea43b4e"
format_stamp: "Formatted at 2026-08-07 04:24:31 on dist-test-slave-zkpd"
server_key: "10034a80355c12d7650a513468e98ca5"
server_key_iv: "d58012f1c7bdcfd15d2cc95002bb98a8"
server_key_version: "kuduclusterkey@0"
I20260807 04:24:31.909288 14007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:24:31.940145 14007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:24:31.944283 14007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:24:31.944620 14007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:24:31.945657 14007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:24:31.947031 14007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:24:31.947137 14007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:31.947299 14007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:24:31.947374 14007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:24:31.983176 14007 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:42161
I20260807 04:24:31.983206 14140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:42161 every 8 connection(s)
I20260807 04:24:31.984557 14007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:24:31.986112 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 14007
I20260807 04:24:31.986338 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:24:31.986781 2003 external_mini_cluster.cc:1428] Setting key 3a2960aa1f7638fd4f207b1e42c3a68f
Aug 07 04:24:31 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076671, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:24:32.004983 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:31.987247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35523 (local address 127.1.244.254:37519)
0807 04:24:31.987930 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:31.987937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:31.988886 (+ 949us) server_negotiation.cc:407] Connection header received
0807 04:24:31.990892 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:31.990896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:31.990980 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:31.991069 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:31.992826 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.993538 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.995122 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.995561 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.995649 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:31.998873 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:31.998878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:31.998881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:31.998914 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:32.001877 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.002573 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.002578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.002588 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.002712 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.003184 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.003188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.003190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.003462 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:32.003621 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.004361 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.004595 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":362,"threads_started":1}
I20260807 04:24:32.005892 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:31.987700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:37519 (local address 127.1.244.195:35523)
0807 04:24:31.988714 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:31.988767 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:31.990660 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:31.991310 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:31.991319 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:31.991826 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:24:31.992655 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:31.992673 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.993736 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.993742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:31.995000 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:31.995012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:31.995733 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:31.995738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:31.996010 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:31.996655 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:24:31.996675 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:24:31.998681 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:24:32.001994 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:24:32.002003 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:24:32.002016 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:24:32.002421 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:24:32.002828 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:24:32.002833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:24:32.002837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:24:32.003054 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:24:32.003601 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:24:32.003609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:24:32.003970 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 04:24:32.004244 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 04:24:32.004759 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":474,"threads_started":1}
I20260807 04:24:32.007210 14141 heartbeater.cc:347] Connected to a master server at 127.1.244.254:37519
I20260807 04:24:32.007514 14141 heartbeater.cc:464] Registering TS with master...
I20260807 04:24:32.008244 14141 heartbeater.cc:511] Master 127.1.244.254:37519 requested a full tablet report, sending...
I20260807 04:24:32.009614 13628 ts_manager.cc:194] Registered new tserver with Master: 1b8cd37af6924bfeb7770f87fea43b4e (127.1.244.195:42161)
I20260807 04:24:32.010421 13628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:35523
I20260807 04:24:32.014338 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:24:32.031389 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.018806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.1.244.254:37519)
0807 04:24:32.018987 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.018992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.019550 (+ 558us) server_negotiation.cc:407] Connection header received
0807 04:24:32.019742 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.019747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.019823 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.019963 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:32.021378 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.022073 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.023381 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.023798 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.023957 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.026231 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:32.026236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:32.026239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:32.026268 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:32.028719 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.029369 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.029376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.029380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.029512 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.029912 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.029918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.029924 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.030182 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:32.030459 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.030771 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.031051 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:24:32.035259 13628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:24:32.042572 13628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:24:32.070338 14160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.055854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42161 (local address 127.0.0.1:49854)
0807 04:24:32.057027 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:32.057050 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:32.057185 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:32.063416 (+ 6231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:32.063420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:32.063448 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:24:32.063999 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.064029 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.065932 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.065937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.067674 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.067691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.069388 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.069397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.069589 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.069646 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 04:24:32.069752 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 04:24:32.069877 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":38,"thread_start_us":906,"threads_started":1}
I20260807 04:24:32.071272 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.056289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49854 (local address 127.1.244.195:42161)
0807 04:24:32.062824 (+ 6535us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.062833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.062864 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:24:32.062983 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.062991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.063199 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.063409 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:24:32.064256 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.065752 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.067864 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.069181 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.070397 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.070685 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.070762 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.070932 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6349,"thread_start_us":6141,"threads_started":1}
I20260807 04:24:32.071410 14161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.056625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35173 (local address 127.0.0.1:52510)
0807 04:24:32.057693 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:32.057713 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:32.057823 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:32.063042 (+ 5219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:32.063050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:32.063081 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:24:32.063601 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.063616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.065754 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.065759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.068325 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.068346 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.070547 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.070555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.070744 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.070776 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:24:32.070869 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:24:32.070998 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":1,"thread_start_us":995,"threads_started":1}
I20260807 04:24:32.071485 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.057406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.1.244.193:35173)
0807 04:24:32.062425 (+ 5019us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.062435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.062466 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:24:32.062566 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.062573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.062837 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.063040 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:24:32.063801 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.065595 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.068559 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.070082 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.070250 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.070539 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.070933 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.071118 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4773,"thread_start_us":4574,"threads_started":1}
I20260807 04:24:32.072032 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.055276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:36639 (local address 127.0.0.1:60892)
0807 04:24:32.056169 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:32.056217 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:32.056384 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:32.063635 (+ 7251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:32.063639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:32.063686 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:24:32.064280 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.064293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.066147 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.066160 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.068424 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.068438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.071077 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.071087 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.071290 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.071328 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:24:32.071403 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 04:24:32.071533 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":60,"spinlock_wait_cycles":8576,"thread_start_us":349,"threads_started":1}
I20260807 04:24:32.073189 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.055445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.1.244.194:36639)
0807 04:24:32.063033 (+ 7588us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.063040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.063065 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:24:32.063160 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.063169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.063410 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.063982 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:24:32.064505 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.065991 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.068592 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.070130 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.072614 (+ 2484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.072802 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.072860 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.072986 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7394,"thread_start_us":7286,"threads_started":1}
I20260807 04:24:32.075526 14074 tablet_service.cc:1467] Processing CreateTablet for tablet 6db4abb383ba4664913eb1a67251d10b (DEFAULT_TABLE table=test-table [id=89a9fa6ee2384b95b21770d7fdc2116f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:24:32.075526 13781 tablet_service.cc:1467] Processing CreateTablet for tablet 6db4abb383ba4664913eb1a67251d10b (DEFAULT_TABLE table=test-table [id=89a9fa6ee2384b95b21770d7fdc2116f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:24:32.076103 13929 tablet_service.cc:1467] Processing CreateTablet for tablet 6db4abb383ba4664913eb1a67251d10b (DEFAULT_TABLE table=test-table [id=89a9fa6ee2384b95b21770d7fdc2116f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:24:32.077086 13781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db4abb383ba4664913eb1a67251d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:24:32.077086 14074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db4abb383ba4664913eb1a67251d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:24:32.077893 13929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db4abb383ba4664913eb1a67251d10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:24:32.091930 14163 tablet_bootstrap.cc:492] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Bootstrap starting.
I20260807 04:24:32.096767 14165 tablet_bootstrap.cc:492] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Bootstrap starting.
I20260807 04:24:32.097304 14163 tablet_bootstrap.cc:654] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Neither blocks nor log segments found. Creating new log.
I20260807 04:24:32.098754 14164 tablet_bootstrap.cc:492] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Bootstrap starting.
I20260807 04:24:32.098790 14163 log.cc:824] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Log is configured to *not* fsync() on all Append() calls
I20260807 04:24:32.099823 14165 tablet_bootstrap.cc:654] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Neither blocks nor log segments found. Creating new log.
I20260807 04:24:32.100929 14165 log.cc:824] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Log is configured to *not* fsync() on all Append() calls
I20260807 04:24:32.102047 14163 tablet_bootstrap.cc:492] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: No bootstrap required, opened a new log
I20260807 04:24:32.102008 14164 tablet_bootstrap.cc:654] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Neither blocks nor log segments found. Creating new log.
I20260807 04:24:32.102723 14163 ts_tablet_manager.cc:1397] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260807 04:24:32.103572 14165 tablet_bootstrap.cc:492] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: No bootstrap required, opened a new log
I20260807 04:24:32.103550 14164 log.cc:824] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:24:32.103843 14165 ts_tablet_manager.cc:1397] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260807 04:24:32.106467 14164 tablet_bootstrap.cc:492] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: No bootstrap required, opened a new log
I20260807 04:24:32.106721 14164 ts_tablet_manager.cc:1397] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260807 04:24:32.111428 14165 raft_consensus.cc:348] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.111794 14165 raft_consensus.cc:374] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:24:32.111959 14165 raft_consensus.cc:729] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c1becafa09040649cbe42eacf54f315, State: Initialized, Role: FOLLOWER
I20260807 04:24:32.112632 14165 consensus_queue.cc:260] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.114354 14165 ts_tablet_manager.cc:1428] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260807 04:24:32.115322 14163 raft_consensus.cc:348] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.115751 14163 raft_consensus.cc:374] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:24:32.115850 14163 raft_consensus.cc:729] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8cd37af6924bfeb7770f87fea43b4e, State: Initialized, Role: FOLLOWER
I20260807 04:24:32.116492 14163 consensus_queue.cc:260] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.117668 14141 heartbeater.cc:503] Master 127.1.244.254:37519 was elected leader, sending a full tablet report...
I20260807 04:24:32.118281 14163 ts_tablet_manager.cc:1428] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260807 04:24:32.118702 14164 raft_consensus.cc:348] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.119143 14164 raft_consensus.cc:374] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:24:32.119256 14164 raft_consensus.cc:729] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c193cecb2747edaa4b8d6004c5d10a, State: Initialized, Role: FOLLOWER
I20260807 04:24:32.119845 14164 consensus_queue.cc:260] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.120992 14001 heartbeater.cc:503] Master 127.1.244.254:37519 was elected leader, sending a full tablet report...
I20260807 04:24:32.121599 14164 ts_tablet_manager.cc:1428] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260807 04:24:32.236220 14142 tablet.cc:2369] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:24:32.240348 14172 raft_consensus.cc:482] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:24:32.240660 14172 raft_consensus.cc:504] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.242393 14172 leader_election.cc:288] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c1becafa09040649cbe42eacf54f315 (127.1.244.193:35173), 1b8cd37af6924bfeb7770f87fea43b4e (127.1.244.195:42161)
I20260807 04:24:32.249784 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.243008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:49659 (local address 127.1.244.195:42161)
0807 04:24:32.243221 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.243227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.243937 (+ 710us) server_negotiation.cc:407] Connection header received
0807 04:24:32.244066 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.244070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.244165 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.244266 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:24:32.244929 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.245866 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.247936 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.248750 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.248916 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.248993 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.249346 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.249527 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:24:32.249912 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.242707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:34089 (local address 127.1.244.193:35173)
0807 04:24:32.242930 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.242936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.243686 (+ 750us) server_negotiation.cc:407] Connection header received
0807 04:24:32.243925 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.243931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.244020 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.244107 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:24:32.244769 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.245732 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.247980 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.248944 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.249107 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.249179 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.249533 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.249682 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:24:32.250000 14174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.242828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:42161 (local address 127.1.244.194:49659)
0807 04:24:32.243737 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:32.243751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:32.243941 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:32.244295 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:32.244299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:32.244327 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:24:32.244746 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.244757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.246021 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.246026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.247769 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.247782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.248969 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.248978 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.249177 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.249212 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:24:32.249297 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 04:24:32.249391 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":22,"thread_start_us":557,"threads_started":1}
I20260807 04:24:32.250067 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.242565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:35173 (local address 127.1.244.194:34089)
0807 04:24:32.243536 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0807 04:24:32.243561 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 04:24:32.243732 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:24:32.244148 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:24:32.244153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:24:32.244177 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:24:32.244613 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.244623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.245873 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.245877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.247775 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:24:32.247787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.249160 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.249165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:24:32.249364 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.249396 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:24:32.249483 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 04:24:32.249597 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":21,"thread_start_us":563,"threads_started":1}
I20260807 04:24:32.251055 14094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db4abb383ba4664913eb1a67251d10b" candidate_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" is_pre_election: true
I20260807 04:24:32.251101 13802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db4abb383ba4664913eb1a67251d10b" candidate_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c1becafa09040649cbe42eacf54f315" is_pre_election: true
I20260807 04:24:32.251580 14094 raft_consensus.cc:2454] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0c193cecb2747edaa4b8d6004c5d10a in term 0.
I20260807 04:24:32.251636 13802 raft_consensus.cc:2454] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0c193cecb2747edaa4b8d6004c5d10a in term 0.
I20260807 04:24:32.252300 13883 leader_election.cc:302] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b8cd37af6924bfeb7770f87fea43b4e, c0c193cecb2747edaa4b8d6004c5d10a; no voters:
I20260807 04:24:32.252731 14172 raft_consensus.cc:2781] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:24:32.252924 14172 raft_consensus.cc:482] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:24:32.253067 14172 raft_consensus.cc:3037] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:24:32.255591 14172 raft_consensus.cc:504] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.256201 14172 leader_election.cc:288] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [CANDIDATE]: Term 1 election: Requested vote from peers 3c1becafa09040649cbe42eacf54f315 (127.1.244.193:35173), 1b8cd37af6924bfeb7770f87fea43b4e (127.1.244.195:42161)
I20260807 04:24:32.256649 14094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db4abb383ba4664913eb1a67251d10b" candidate_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b8cd37af6924bfeb7770f87fea43b4e"
I20260807 04:24:32.256719 13802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db4abb383ba4664913eb1a67251d10b" candidate_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c1becafa09040649cbe42eacf54f315"
I20260807 04:24:32.256832 14094 raft_consensus.cc:3037] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:24:32.256930 13802 raft_consensus.cc:3037] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:24:32.258992 14094 raft_consensus.cc:2454] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0c193cecb2747edaa4b8d6004c5d10a in term 1.
I20260807 04:24:32.259204 13802 raft_consensus.cc:2454] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0c193cecb2747edaa4b8d6004c5d10a in term 1.
I20260807 04:24:32.259410 13883 leader_election.cc:302] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b8cd37af6924bfeb7770f87fea43b4e, c0c193cecb2747edaa4b8d6004c5d10a; no voters:
I20260807 04:24:32.259719 14172 raft_consensus.cc:2781] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:24:32.260172 14172 raft_consensus.cc:686] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 1 LEADER]: Becoming Leader. State: Replica: c0c193cecb2747edaa4b8d6004c5d10a, State: Running, Role: LEADER
I20260807 04:24:32.260618 14172 consensus_queue.cc:237] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } }
I20260807 04:24:32.265277 13626 catalog_manager.cc:5495] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a reported cstate change: term changed from 0 to 1, leader changed from <none> to c0c193cecb2747edaa4b8d6004c5d10a (127.1.244.194). New cstate: current_term: 1 leader_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0c193cecb2747edaa4b8d6004c5d10a" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 36639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 } health_report { overall_health: UNKNOWN } } }
I20260807 04:24:32.304435 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.298563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60908 (local address 127.1.244.194:36639)
0807 04:24:32.298803 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.298811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.298829 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:24:32.299209 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.299214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.299305 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.299463 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:24:32.299877 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.300655 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.302173 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.302572 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.302724 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.303276 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:24:32.303760 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:24:32.303964 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.304072 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.304151 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:24:32.312358 13802 raft_consensus.cc:1261] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 1 FOLLOWER]: Refusing update from remote peer c0c193cecb2747edaa4b8d6004c5d10a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:24:32.312367 14094 raft_consensus.cc:1261] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 1 FOLLOWER]: Refusing update from remote peer c0c193cecb2747edaa4b8d6004c5d10a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:24:32.313342 14175 consensus_queue.cc:1035] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b8cd37af6924bfeb7770f87fea43b4e" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 42161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:24:32.313669 14172 consensus_queue.cc:1035] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c1becafa09040649cbe42eacf54f315" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 35173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:24:32.320221 13850 tablet.cc:2369] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:24:32.327387 14180 mvcc.cc:205] Tried to move back new op lower bound from 7315770049781608448 to 7315770049583702016. Current Snapshot: MvccSnapshot[applied={T|T < 7315770049781608448}]
I20260807 04:24:32.352919 13627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:24:32.353277 13627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:24:32.358476 13627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88422a3d0a3249a7a33de5ff198f79c9: Sending DeleteTablet for 3 replicas of tablet 6db4abb383ba4664913eb1a67251d10b
I20260807 04:24:32.360167 14074 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db4abb383ba4664913eb1a67251d10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:24:32 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:49854
I20260807 04:24:32.360852 13781 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db4abb383ba4664913eb1a67251d10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:24:32 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:52510
I20260807 04:24:32.361205 14187 tablet_replica.cc:330] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: stopping tablet replica
I20260807 04:24:32.361610 14187 raft_consensus.cc:2229] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:24:32.362162 14187 raft_consensus.cc:2258] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:24:32.362311 13929 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db4abb383ba4664913eb1a67251d10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:24:32 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:60892
I20260807 04:24:32.362829 14073 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db4abb383ba4664913eb1a67251d10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:24:32 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:49854
I20260807 04:24:32.364285 14187 ts_tablet_manager.cc:1905] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:24:32.364842 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 13711
I20260807 04:24:32.370010 14189 tablet_replica.cc:330] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: stopping tablet replica
I20260807 04:24:32.370458 14189 raft_consensus.cc:2229] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:24:32.371212 14189 raft_consensus.cc:2258] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:24:32.371512 14187 ts_tablet_manager.cc:1918] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:24:32.371743 14187 log.cc:1196] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/6db4abb383ba4664913eb1a67251d10b
I20260807 04:24:32.372354 14187 ts_tablet_manager.cc:1939] T 6db4abb383ba4664913eb1a67251d10b P 1b8cd37af6924bfeb7770f87fea43b4e: Deleting consensus metadata
I20260807 04:24:32.372619 14188 tablet_replica.cc:330] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: stopping tablet replica
I20260807 04:24:32.373071 14188 raft_consensus.cc:2229] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:24:32.373626 14188 raft_consensus.cc:2258] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:24:32.374327 13614 catalog_manager.cc:4841] TS 1b8cd37af6924bfeb7770f87fea43b4e (127.1.244.195:42161): tablet 6db4abb383ba4664913eb1a67251d10b (table test-table [id=89a9fa6ee2384b95b21770d7fdc2116f]) successfully deleted
W20260807 04:24:32.375382 13614 catalog_manager.cc:4808] TS 1b8cd37af6924bfeb7770f87fea43b4e (127.1.244.195:42161): delete failed for tablet 6db4abb383ba4664913eb1a67251d10b because the tablet was not found. No further retry: Not found: Tablet not found: 6db4abb383ba4664913eb1a67251d10b
I20260807 04:24:32.375864 14188 ts_tablet_manager.cc:1905] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:24:32.375874 14189 ts_tablet_manager.cc:1905] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:24:32.383309 14189 ts_tablet_manager.cc:1918] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:24:32.383394 14188 ts_tablet_manager.cc:1918] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:24:32.383594 14189 log.cc:1196] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/6db4abb383ba4664913eb1a67251d10b
I20260807 04:24:32.383602 14188 log.cc:1196] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/6db4abb383ba4664913eb1a67251d10b
I20260807 04:24:32.384222 14188 ts_tablet_manager.cc:1939] T 6db4abb383ba4664913eb1a67251d10b P 3c1becafa09040649cbe42eacf54f315: Deleting consensus metadata
I20260807 04:24:32.384214 14189 ts_tablet_manager.cc:1939] T 6db4abb383ba4664913eb1a67251d10b P c0c193cecb2747edaa4b8d6004c5d10a: Deleting consensus metadata
Aug 07 04:24:32 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:24:32.386058 13612 catalog_manager.cc:4841] TS 3c1becafa09040649cbe42eacf54f315 (127.1.244.193:35173): tablet 6db4abb383ba4664913eb1a67251d10b (table test-table [id=89a9fa6ee2384b95b21770d7fdc2116f]) successfully deleted
I20260807 04:24:32.386479 13612 catalog_manager.cc:4841] TS c0c193cecb2747edaa4b8d6004c5d10a (127.1.244.194:36639): tablet 6db4abb383ba4664913eb1a67251d10b (table test-table [id=89a9fa6ee2384b95b21770d7fdc2116f]) successfully deleted
I20260807 04:24:32.393204 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.366401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.1.244.193:35173)
0807 04:24:32.366570 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.366577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.371967 (+ 5390us) server_negotiation.cc:407] Connection header received
0807 04:24:32.372167 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.372175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.372289 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.372415 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:32.376722 (+ 4307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.377901 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.380022 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.380721 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.380823 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.387127 (+ 6304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:32.387139 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:32.387152 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:32.387188 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:32.390237 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.391154 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.391161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.391163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.391290 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.391735 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.391738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.391740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.392183 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:32.392400 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.392761 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.392935 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 04:24:32.394057 13844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:24:32.484699 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 13711
I20260807 04:24:32.509331 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 13855
Aug 07 04:24:32 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:24:32.527616 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.510285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.1.244.194:36639)
0807 04:24:32.510457 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.510462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.510480 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:24:32.510584 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.510587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.510648 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.510731 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:32.512529 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.513431 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.515464 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.515870 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.516022 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.521339 (+ 5317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:32.521352 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:32.521369 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:32.521405 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:32.524783 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.525438 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.525446 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.525448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.525588 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.526031 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.526034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.526036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.526391 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:32.526754 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.527009 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.527231 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:24:32.528508 13994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:24:32.646061 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 13855
I20260807 04:24:32.668807 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 14007
Aug 07 04:24:32 dist-test-slave-zkpd krb5kdc[13136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:24:32.683862 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:24:32.669622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.1.244.195:42161)
0807 04:24:32.669841 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 04:24:32.669845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:24:32.669863 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:24:32.669965 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:24:32.669969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:24:32.670030 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:24:32.670124 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:24:32.671377 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.672258 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.673468 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:24:32.673877 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:24:32.673970 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:24:32.678147 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:24:32.678160 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:24:32.678174 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:24:32.678205 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:24:32.681249 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.681851 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.681858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.681860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.681987 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:24:32.682458 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:24:32.682461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:24:32.682463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:24:32.682830 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:24:32.683048 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:24:32.683306 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 04:24:32.683459 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 04:24:32.684595 14136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:24:32.773828 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 14007
I20260807 04:24:32.792620 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 13596
I20260807 04:24:32.793974 13659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:24:32.905532 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 13596
I20260807 04:24:32.929225 2003 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 04:24:33.265472 2003 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 04:24:33.265666 2003 mini_ranger.cc:66] Stopping Ranger...
I20260807 04:24:33.611181 2003 mini_ranger.cc:68] Stopped Ranger
2026-08-07 04:24:33.611 UTC [13168] LOG: received smart shutdown request
2026-08-07 04:24:33.615 UTC [13168] LOG: background worker "logical replication launcher" (PID 13177) exited with exit code 1
2026-08-07 04:24:33.615 UTC [13172] LOG: shutting down
2026-08-07 04:24:33.768 UTC [13168] LOG: database system is shut down
2026-08-07T04:24:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47990 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:24:34 dist-test-slave-zkpd krb5kdc[14203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:24:34 dist-test-slave-zkpd krb5kdc[14203](info): set up 2 sockets
Aug 07 04:24:34 dist-test-slave-zkpd krb5kdc[14203](info): commencing operation
krb5kdc: starting...
W20260807 04:24:35.018827 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:24:35 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:24:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:24:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/rangeradmin_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/rangerlookup_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
I20260807 04:24:35.372740 2003 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskA8s8yC/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/postgres -l logfile start
2026-08-07 04:24:37.750 UTC [14234] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 04:24:37.750 UTC [14234] LOG: listening on IPv4 address "127.1.244.212", port 54047
2026-08-07 04:24:37.754 UTC [14234] LOG: listening on Unix socket "/tmp/.s.PGSQL.54047"
2026-08-07 04:24:37.777 UTC [14237] LOG: database system was shut down at 2026-08-07 04:24:36 UTC
2026-08-07 04:24:37.780 UTC [14234] LOG: database system is ready to accept connections
I20260807 04:24:38.758590 2003 mini_postgres.cc:96] Postgres bound to 54047
2026-08-07 04:24:38.780 UTC [14245] FATAL: database "slave" does not exist
127.1.244.212:54047 - accepting connections
I20260807 04:24:38.781247 2003 mini_ranger.cc:161] Starting Ranger...
I20260807 04:24:38.810817 2003 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 04:24:39.259505 2003 mini_ranger.cc:87] Created ranger Postgres database
I20260807 04:24:39.259635 2003 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-admin
2026-08-07 04:24:39,538 [I] DB FLAVOR :POSTGRES
2026-08-07 04:24:39,538 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:24:39,538 [I] Checking connection..
2026-08-07 04:24:39,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 04:24:39,792 [I] Checking connection passed.
2026-08-07 04:24:39,792 [I] --------- Verifying version history table ---------
2026-08-07 04:24:39,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:24:40,051 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:24:40,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:24:40,284 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 04:24:40,284 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 04:24:40,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 04:24:40.536 UTC [14314] WARNING: there is no transaction in progress
2026-08-07 04:24:40,543 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 04:24:40,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 04:24:40,781 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 04:24:40,781 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:24:40,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 04:24:41,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 04:24:41,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-07 04:24:41,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:24:41,763 [I] Table x_portal_user does not exist in database ranger
2026-08-07 04:24:41,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:24:41,990 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 04:24:41,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:24:42,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:24:42,473 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 04:24:42,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 04:24:43.402 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.417 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.434 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.449 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.468 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.486 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.538 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.544 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.557 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.567 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.576 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.589 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.599 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.605 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.611 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:43.618 UTC [14427] WARNING: there is no transaction in progress
2026-08-07 04:24:44,212 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 04:24:44,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 04:24:44,464 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 04:24:44,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 04:24:44,718 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 04:24:44,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 04:24:44,960 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 04:24:44,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 04:24:45,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 04:24:45,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-07 04:24:45,661 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 04:24:45,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 04:24:45,888 [I] DB_PATCHES have already been applied
I20260807 04:24:45.894500 2003 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-admin:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskA8s8yC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 04:24:45.894586 2003 mini_ranger.cc:193] Using host: 127.1.244.212
I20260807 04:24:45.913103 2003 mini_ranger.cc:227] Ranger admin URL: http://127.1.244.212:56171
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 4:24:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 04:24:47.051563 2003 mini_ranger.cc:160] Time spent starting Ranger: real 8.270s user 0.004s sys 0.045s
Aug 07, 2026 4:25:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 4:25:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 4:25:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 4:25:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 4:25:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 04:25:09.015420 2003 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.244.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.244.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.244.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/rangerkms_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.244.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.244.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.244.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/HTTP_127.1.244.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 04:25:09 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076709, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 04:25:09.219233 2003 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 04:25:09.242352 2003 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 04:25:10.125041 2003 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 04:25:10.125138 2003 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms
I20260807 04:25:10.125169 2003 mini_ranger_kms.cc:227] Using postgres at 127.1.244.212:54047
2026-08-07 04:25:10,486 [I] DB FLAVOR :POSTGRES
2026-08-07 04:25:10,486 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 04:25:10,486 [I] Checking connection
2026-08-07 04:25:10,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 04:25:10,719 [I] connection success
2026-08-07 04:25:10,719 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 04:25:10,720 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 04:25:10,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 04:25:10,953 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 04:25:10,953 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 04:25:10,953 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 04:25:10,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.244.212:54047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 04:25:11,234 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 04:25:11.717608 2003 mini_ranger_kms.cc:310] Created kms service
I20260807 04:25:11.907208 2003 mini_ranger_kms.cc:326] Created kudu user
I20260807 04:25:11.953104 2003 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 04:25:11 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076709, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:12.353838 2003 mini_ranger_kms.cc:384] Added ranger policy
I20260807 04:25:12.354183 2003 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms:/tmp/dist-test-taskA8s8yC/build/asan/bin/postgresql.jar:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskA8s8yC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 04:25:12.354247 2003 mini_ranger_kms.cc:240] Using host: 127.1.244.212
I20260807 04:25:12.373945 2003 mini_ranger_kms.cc:276] Ranger KMS PID: 14617
I20260807 04:25:12.374081 2003 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.244.212:58923
Aug 07, 2026 4:25:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 4:25:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 4:25:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 4:25:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 4:25:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 4:25:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 04:25:14 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786076714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:14.710752 2003 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.492s user 0.080s sys 0.304s
I20260807 04:25:14.711018 2003 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 04:25:14.711226 2003 mini_ranger_kms.cc:401] 127.1.244.212:58923/kms/v1/keys
Aug 07 04:25:14 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Aug 07 04:25:15 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076714, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.244.212@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Aug 07 04:25:17 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:25:17.565672 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40123
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40123
--ranger_config_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:17.916532 14668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:17.916812 14668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:17.916900 14668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:17.923977 14668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:17.924082 14668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:17.924129 14668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:17.924185 14668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:17.924234 14668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:17.933984 14668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.244.212:58923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40123
--ranger_config_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:17.935140 14668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:17.936388 14668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:17.948932 14676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:17.948990 14674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:17.949018 14673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:17.949877 14668 server_base.cc:1034] running on GCE node
I20260807 04:25:17.950853 14668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:17.952262 14668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:17.953490 14668 hybrid_clock.cc:648] HybridClock initialized: now 1786076717953452 us; error 44 us; skew 500 ppm
I20260807 04:25:17.953868 14668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:17 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:17.956944 14668 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:17.958382 14668 webserver.cc:460] Webserver started at http://127.1.244.254:44035/ using document root <none> and password file <none>
I20260807 04:25:17.959038 14668 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:17.959177 14668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:17.959496 14668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:25:17 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:18.020586 14668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "b5a6a890e2ad47479882dd07e1071ca7"
format_stamp: "Formatted at 2026-08-07 04:25:18 on dist-test-slave-zkpd"
server_key: "d0644d51ae359e2d378dfba39ed24612"
server_key_iv: "12fe36504150396236af1446fd5c2e21"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:18.021710 14668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "b5a6a890e2ad47479882dd07e1071ca7"
format_stamp: "Formatted at 2026-08-07 04:25:18 on dist-test-slave-zkpd"
server_key: "d0644d51ae359e2d378dfba39ed24612"
server_key_iv: "12fe36504150396236af1446fd5c2e21"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:18.028762 14668 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260807 04:25:18.049422 14683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:18.051186 14668 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:25:18.051388 14668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "b5a6a890e2ad47479882dd07e1071ca7"
format_stamp: "Formatted at 2026-08-07 04:25:18 on dist-test-slave-zkpd"
server_key: "d0644d51ae359e2d378dfba39ed24612"
server_key_iv: "12fe36504150396236af1446fd5c2e21"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:18.051755 14668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:18.082192 14668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:18.087289 14668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:18.087625 14668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:18.107846 14668 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40123
I20260807 04:25:18.107872 14738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40123 every 8 connection(s)
I20260807 04:25:18.109233 14668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:18.113106 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 14668
I20260807 04:25:18.113134 14739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:18.113335 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:25:18.114070 2003 external_mini_cluster.cc:1428] Setting key fa4e677b841fb4071da7d189b4f86c38
I20260807 04:25:18.126570 14739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Bootstrap starting.
Aug 07 04:25:18 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:18.130146 14739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:18.131224 14739 log.cc:824] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:18.134516 14739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: No bootstrap required, opened a new log
I20260807 04:25:18.138103 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:18.116113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.1.244.254:40123)
0807 04:25:18.116903 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:18.116918 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:18.116958 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:25:18.118977 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:18.118999 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:18.119410 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:18.119717 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:18.121366 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:18.122639 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:18.124619 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:18.125281 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:18.125418 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:18.129958 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:18.129966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:18.129980 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:18.130015 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:18.133596 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:18.134285 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:18.134294 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:18.134301 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:18.134509 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:18.134922 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:18.134926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:18.134931 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:18.135617 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:18.135942 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:18.136337 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 04:25:18.136775 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":392,"threads_started":1}
I20260807 04:25:18.144136 14739 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:18.144467 14739 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:18.144541 14739 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a6a890e2ad47479882dd07e1071ca7, State: Initialized, Role: FOLLOWER
I20260807 04:25:18.145072 14739 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:18.145272 14739 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:25:18.145408 14739 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:25:18.145545 14739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:18.148002 14739 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:18.148425 14739 leader_election.cc:302] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5a6a890e2ad47479882dd07e1071ca7; no voters:
I20260807 04:25:18.148962 14739 leader_election.cc:288] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:25:18.149107 14745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:18.149793 14745 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 LEADER]: Becoming Leader. State: Replica: b5a6a890e2ad47479882dd07e1071ca7, State: Running, Role: LEADER
I20260807 04:25:18.150357 14745 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:18.150961 14739 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:18.153158 14746 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5a6a890e2ad47479882dd07e1071ca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } } }
I20260807 04:25:18.153611 14746 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:18.154270 14748 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5a6a890e2ad47479882dd07e1071ca7. Latest consensus state: current_term: 1 leader_uuid: "b5a6a890e2ad47479882dd07e1071ca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } } }
I20260807 04:25:18.154769 14748 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:18.156798 14755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:18.162401 14755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:18.163172 14739 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 04:25:18.180028 14755 catalog_manager.cc:1347] Generated new cluster ID: cc1fe4ac76d4499aad8c793b1c27f1e0
I20260807 04:25:18.180189 14755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:25:18.196466 14755 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:25:18.197830 14755 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:25:18.208909 14755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Generated new TSK 0
I20260807 04:25:18.209997 14755 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:25:18.936362 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40123
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:25:19 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786076719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 04:25:19.375262 14787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:19.375758 14787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:19.375984 14787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:19.387660 14787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:19.387884 14787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:19.388147 14787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:25:19.402016 14787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.244.212:58923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:19.403368 14787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:19.404743 14787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:19.417969 14797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:19.418258 14800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:19.418552 14798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:19.420820 14787 server_base.cc:1034] running on GCE node
I20260807 04:25:19.421459 14787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:19.422564 14787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:19.423799 14787 hybrid_clock.cc:648] HybridClock initialized: now 1786076719423753 us; error 39 us; skew 500 ppm
I20260807 04:25:19.424140 14787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:19 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:19.436040 14787 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:25:19.443819 14787 webserver.cc:460] Webserver started at http://127.1.244.193:46213/ using document root <none> and password file <none>
I20260807 04:25:19.444789 14787 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:19.444933 14787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:19.445274 14787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:25:19 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:19.475827 14787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "d006b1d2635a4315a18ca9fe8948258d"
format_stamp: "Formatted at 2026-08-07 04:25:19 on dist-test-slave-zkpd"
server_key: "169df0bc946ad3ca74e7088b6f55427b"
server_key_iv: "4d12c98f3766ff5bf25f3191016e5290"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:19.476960 14787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "d006b1d2635a4315a18ca9fe8948258d"
format_stamp: "Formatted at 2026-08-07 04:25:19 on dist-test-slave-zkpd"
server_key: "169df0bc946ad3ca74e7088b6f55427b"
server_key_iv: "4d12c98f3766ff5bf25f3191016e5290"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:19.493705 14787 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.007s sys 0.007s
I20260807 04:25:19.518764 14807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:19.520690 14787 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260807 04:25:19.520907 14787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "d006b1d2635a4315a18ca9fe8948258d"
format_stamp: "Formatted at 2026-08-07 04:25:19 on dist-test-slave-zkpd"
server_key: "169df0bc946ad3ca74e7088b6f55427b"
server_key_iv: "4d12c98f3766ff5bf25f3191016e5290"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:19.521178 14787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:19.598657 14787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:19.604262 14787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:19.604687 14787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:19.612104 14787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:19.620262 14787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:19.620424 14787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:19.620621 14787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:19.620743 14787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 04:25:19 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Aug 07 04:25:20 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:20.162379 14787 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36577
I20260807 04:25:20.162397 14929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36577 every 8 connection(s)
I20260807 04:25:20.164075 14787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:25:20.170785 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 14787
I20260807 04:25:20.171058 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:25:20.172156 2003 external_mini_cluster.cc:1428] Setting key 3cb7da96be40f9e05ecd22a1457f6851
Aug 07 04:25:20 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076719, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:20.201289 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:20.168767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:58755 (local address 127.1.244.254:40123)
0807 04:25:20.180932 (+ 12165us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:20.180937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:20.180955 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:25:20.181009 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:20.181017 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:20.181096 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:20.181203 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:20.184292 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.185452 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.187841 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.188488 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.188583 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:20.193232 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:20.193239 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:20.193244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:20.193283 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:20.197197 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:20.198203 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:20.198209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:20.198213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:20.198371 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:20.199005 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:20.199010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:20.199014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:20.199386 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:20.199538 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:20.200649 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0807 04:25:20.200905 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12042,"thread_start_us":11885,"threads_started":1}
I20260807 04:25:20.204006 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:20.168126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40123 (local address 127.1.244.193:58755)
0807 04:25:20.176529 (+ 8403us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:20.176591 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:20.180044 (+ 3453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:20.182503 (+ 2459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:20.182517 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:20.183190 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:20.184070 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:20.184098 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.185662 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.185667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:20.187655 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:20.187670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.188952 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.188958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:20.189235 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:20.190082 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:20.190112 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:20.192974 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:20.197407 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:20.197416 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:20.197435 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:20.198054 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:20.198525 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:20.198530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:20.198533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:20.198870 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:20.199637 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:20.199645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:20.200160 (+ 515us) client_negotiation.cc:769] Sending connection context
0807 04:25:20.201884 (+ 1724us) client_negotiation.cc:240] Negotiation successful
0807 04:25:20.202450 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8092,"thread_start_us":7831,"threads_started":1}
I20260807 04:25:20.206327 14930 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40123
I20260807 04:25:20.206674 14930 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:20.207551 14930 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:20.210297 14701 ts_manager.cc:194] Registered new tserver with Master: d006b1d2635a4315a18ca9fe8948258d (127.1.244.193:36577)
I20260807 04:25:20.212420 14701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:58755
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:25:20.272238 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40123
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:20.642813 14936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:20.643066 14936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:20.643162 14936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:20.650801 14936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:20.650904 14936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:20.651013 14936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:25:20.660828 14936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.244.212:58923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:20.661979 14936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:20.663173 14936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:20.676388 14949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:20.676359 14950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:20.676378 14952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:20.678210 14936 server_base.cc:1034] running on GCE node
I20260807 04:25:20.678954 14936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:20.679929 14936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:20.681177 14936 hybrid_clock.cc:648] HybridClock initialized: now 1786076720681137 us; error 27 us; skew 500 ppm
I20260807 04:25:20.681504 14936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:20 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076720, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:20.684635 14936 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:25:20.686012 14936 webserver.cc:460] Webserver started at http://127.1.244.194:44499/ using document root <none> and password file <none>
I20260807 04:25:20.686676 14936 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:20.686833 14936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:20.687395 14936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:25:20 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076720, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:20.705973 14936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "7595fd5f5c824b36bd514f7a567cd297"
format_stamp: "Formatted at 2026-08-07 04:25:20 on dist-test-slave-zkpd"
server_key: "5edd4f6b7935c8d72776990dcbe1cb81"
server_key_iv: "8075a3e172ece89e21da6918f0fd868f"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:20.706933 14936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "7595fd5f5c824b36bd514f7a567cd297"
format_stamp: "Formatted at 2026-08-07 04:25:20 on dist-test-slave-zkpd"
server_key: "5edd4f6b7935c8d72776990dcbe1cb81"
server_key_iv: "8075a3e172ece89e21da6918f0fd868f"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:20.712278 14936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 04:25:20.728022 14959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:20.729516 14936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:25:20.729701 14936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "7595fd5f5c824b36bd514f7a567cd297"
format_stamp: "Formatted at 2026-08-07 04:25:20 on dist-test-slave-zkpd"
server_key: "5edd4f6b7935c8d72776990dcbe1cb81"
server_key_iv: "8075a3e172ece89e21da6918f0fd868f"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:20.729936 14936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:20.742273 14936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:20.746207 14936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:20.746497 14936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:20.747412 14936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:20.748916 14936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:20.749022 14936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:20.749166 14936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:20.749238 14936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:20.786932 14936 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45501
I20260807 04:25:20.786988 15076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45501 every 8 connection(s)
I20260807 04:25:20.788250 14936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:25:20.796293 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 14936
I20260807 04:25:20.796517 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:25:20.797045 2003 external_mini_cluster.cc:1428] Setting key 74f76541531fe2fd0d5cb327e1cbe1ab
Aug 07 04:25:20 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076720, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:20.810263 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:20.791128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:60081 (local address 127.1.244.254:40123)
0807 04:25:20.791688 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:20.791694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:20.792754 (+ 1060us) server_negotiation.cc:407] Connection header received
0807 04:25:20.794547 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:20.794552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:20.794644 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:20.794765 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:20.796347 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.797277 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.798873 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.799342 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.799447 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:20.803523 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:20.803532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:20.803540 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:20.803600 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:20.807166 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:20.807973 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:20.807981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:20.807986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:20.808135 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:20.808542 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:20.808549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:20.808555 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:20.808929 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:20.809078 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:20.809748 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 04:25:20.809998 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":277,"threads_started":1}
I20260807 04:25:20.811602 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:20.791661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40123 (local address 127.1.244.194:60081)
0807 04:25:20.792574 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:20.792619 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:20.794265 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:20.795015 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:20.795023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:20.795449 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:20.796156 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:20.796183 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.797452 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.797457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:20.798729 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:20.798749 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:20.799578 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:20.799588 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:20.799954 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:20.800701 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:20.800734 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:20.803298 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:20.807375 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:20.807382 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:20.807393 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:20.807817 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:20.808241 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:20.808244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:20.808246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:20.808426 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:20.809044 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:20.809050 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:20.809357 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 04:25:20.809587 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 04:25:20.810048 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":454,"threads_started":1}
I20260807 04:25:20.813009 15077 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40123
I20260807 04:25:20.813311 15077 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:20.814018 15077 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:20.815527 14701 ts_manager.cc:194] Registered new tserver with Master: 7595fd5f5c824b36bd514f7a567cd297 (127.1.244.194:45501)
I20260807 04:25:20.816380 14701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:60081
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:25:20.856994 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40123
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:21.189266 15083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:21.189584 15083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:21.189774 15083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:21.197157 15083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:21.197288 15083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:21.197444 15083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:25:21.206614 15083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.244.212:58923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:21.207814 15083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:21.209116 15083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 04:25:21.216387 14930 heartbeater.cc:503] Master 127.1.244.254:40123 was elected leader, sending a full tablet report...
W20260807 04:25:21.222136 15088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:21.222182 15089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:21.222333 15091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:21.223258 15083 server_base.cc:1034] running on GCE node
I20260807 04:25:21.223857 15083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:21.224959 15083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:21.226181 15083 hybrid_clock.cc:648] HybridClock initialized: now 1786076721226127 us; error 38 us; skew 500 ppm
I20260807 04:25:21.226509 15083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:21 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076721, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:21.229727 15083 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:25:21.231170 15083 webserver.cc:460] Webserver started at http://127.1.244.195:39111/ using document root <none> and password file <none>
I20260807 04:25:21.231856 15083 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:21.232069 15083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:21.232360 15083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 04:25:21 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076721, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:21.246521 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "f0cad0ae534343379d439df05c60f468"
format_stamp: "Formatted at 2026-08-07 04:25:21 on dist-test-slave-zkpd"
server_key: "6d929c9645ab39483820f48f7ef66c2f"
server_key_iv: "0b8655be58b33f42bf75227941741e84"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:21.247339 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "f0cad0ae534343379d439df05c60f468"
format_stamp: "Formatted at 2026-08-07 04:25:21 on dist-test-slave-zkpd"
server_key: "6d929c9645ab39483820f48f7ef66c2f"
server_key_iv: "0b8655be58b33f42bf75227941741e84"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:21.252897 15083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260807 04:25:21.267554 15098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:21.269066 15083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:25:21.269248 15083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "f0cad0ae534343379d439df05c60f468"
format_stamp: "Formatted at 2026-08-07 04:25:21 on dist-test-slave-zkpd"
server_key: "6d929c9645ab39483820f48f7ef66c2f"
server_key_iv: "0b8655be58b33f42bf75227941741e84"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:21.269497 15083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:21.293051 15083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:21.297025 15083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:21.297320 15083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:21.298242 15083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:21.299554 15083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:21.299625 15083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:21.299798 15083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:21.299849 15083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:21.334177 15083 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:37895
I20260807 04:25:21.334239 15218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:37895 every 8 connection(s)
I20260807 04:25:21.335484 15083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:25:21.341468 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15083
I20260807 04:25:21.341661 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:25:21.342187 2003 external_mini_cluster.cc:1428] Setting key 47b8b6bc6f811362120adea554dc4605
Aug 07 04:25:21 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076721, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:21.354720 15221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:21.338011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:43635 (local address 127.1.244.254:40123)
0807 04:25:21.338493 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:21.338498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:21.339487 (+ 989us) server_negotiation.cc:407] Connection header received
0807 04:25:21.341343 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:21.341349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:21.341461 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:21.341590 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:21.343066 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.343895 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.345744 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.346155 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.346250 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:21.349430 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:21.349435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:21.349439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:21.349469 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:21.352066 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.352796 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.352800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.352802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.352936 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.353313 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.353316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.353318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.353565 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:21.353674 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:21.354304 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 04:25:21.354464 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":227,"threads_started":1}
I20260807 04:25:21.355609 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:21.338621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40123 (local address 127.1.244.195:43635)
0807 04:25:21.339321 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:21.339365 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:21.341078 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:21.341821 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:21.341829 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:21.342261 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:21.342874 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:21.342900 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.344108 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.344113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:21.345593 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:21.345619 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.346336 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.346341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:21.346552 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:21.347188 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:21.347213 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:21.349231 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:21.352229 (+ 2998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:21.352240 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:21.352253 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:21.352679 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:21.353030 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:21.353034 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:21.353038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:21.353213 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:21.353708 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:21.353714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:21.354016 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 04:25:21.354239 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 04:25:21.354591 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":280,"threads_started":1}
I20260807 04:25:21.356884 15219 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40123
I20260807 04:25:21.357223 15219 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:21.357998 15219 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:21.359272 14701 ts_manager.cc:194] Registered new tserver with Master: f0cad0ae534343379d439df05c60f468 (127.1.244.195:37895)
I20260807 04:25:21.360138 14701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:43635
I20260807 04:25:21.369536 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:25:21.371223 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 14787
Aug 07 04:25:21 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:25:21.387033 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:21.372189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53724 (local address 127.1.244.193:36577)
0807 04:25:21.372787 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:21.372793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:21.372835 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:25:21.372911 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:21.372917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:21.373092 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:21.373231 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:21.374750 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.375750 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.377303 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.377832 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.377941 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:21.381994 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:21.382005 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:21.382016 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:21.382046 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:21.384549 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.385208 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.385214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.385217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.385354 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.385660 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.385663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.385665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.385958 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:21.386209 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:21.386462 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 04:25:21.386642 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":312,"threads_started":1}
I20260807 04:25:21.388211 14925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:21.474059 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 14787
I20260807 04:25:21.491865 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 14936
Aug 07 04:25:21 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:25:21.508371 15226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:21.492850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40300 (local address 127.1.244.194:45501)
0807 04:25:21.493498 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:21.493505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:21.493531 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:25:21.493593 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:21.493599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:21.493913 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:21.494045 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:21.495281 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.496425 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.497910 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.498448 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.498552 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:21.502608 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:21.502620 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:21.502634 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:21.502664 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:21.505856 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.506506 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.506513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.506515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.506645 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.506944 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.506947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.506949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.507261 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:21.507476 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:21.507719 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 04:25:21.507978 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":375,"threads_started":1}
I20260807 04:25:21.509677 15072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:21.590441 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 14936
I20260807 04:25:21.612654 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15083
Aug 07 04:25:21 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:25:21.628425 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:21.613701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.1.244.195:37895)
0807 04:25:21.614308 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:21.614314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:21.614342 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:25:21.614399 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:21.614416 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:21.614629 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:21.614758 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:21.616013 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.617145 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.618383 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:21.618914 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:21.619006 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:21.623191 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:21.623202 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:21.623216 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:21.623249 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:21.626086 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.626678 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.626685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.626687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.626810 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:21.627110 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:21.627112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:21.627114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:21.627434 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:21.627669 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:21.627879 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 04:25:21.628129 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":308,"threads_started":1}
I20260807 04:25:21.629705 15214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:21.711781 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15083
I20260807 04:25:21.729496 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 14668
I20260807 04:25:21.730706 14734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:21.836933 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 14668
I20260807 04:25:21.861881 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40123
--webserver_interface=127.1.244.254
--webserver_port=44035
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40123
--ranger_config_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:22.191423 15231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:22.191838 15231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:22.192042 15231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:22.200256 15231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:22.200377 15231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:22.200474 15231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:22.200564 15231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:22.200644 15231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:22.210773 15231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40123
--ranger_config_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:22.212092 15231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:22.213452 15231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:22.228061 15236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:22.228080 15237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:22.228075 15239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:22.228974 15231 server_base.cc:1034] running on GCE node
I20260807 04:25:22.229729 15231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:22.231125 15231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:22.232347 15231 hybrid_clock.cc:648] HybridClock initialized: now 1786076722232308 us; error 50 us; skew 500 ppm
I20260807 04:25:22.232686 15231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:22 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076722, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:22.235759 15231 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:22.237217 15231 webserver.cc:460] Webserver started at http://127.1.244.254:44035/ using document root <none> and password file <none>
I20260807 04:25:22.237849 15231 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:22.237993 15231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:25:22 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076722, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:22.292908 15231 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 04:25:22.296079 15247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:22.297453 15231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:25:22.297643 15231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "b5a6a890e2ad47479882dd07e1071ca7"
format_stamp: "Formatted at 2026-08-07 04:25:18 on dist-test-slave-zkpd"
server_key: "d0644d51ae359e2d378dfba39ed24612"
server_key_iv: "12fe36504150396236af1446fd5c2e21"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:22.298604 15231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:22.319391 15231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:22.323026 15231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:22.323320 15231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:22.342171 15231 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40123
I20260807 04:25:22.342191 15300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40123 every 8 connection(s)
I20260807 04:25:22.343618 15231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:22.347982 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15231
I20260807 04:25:22.348845 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:36577
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:40123
--webserver_port=46213
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:22.350483 15301 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:25:22.353809 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Bootstrap starting.
I20260807 04:25:22.372460 15301 log.cc:824] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:22.381459 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:25:22.381982 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Bootstrap complete.
I20260807 04:25:22.391017 15301 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:22.391952 15301 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a6a890e2ad47479882dd07e1071ca7, State: Initialized, Role: FOLLOWER
I20260807 04:25:22.392567 15301 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:22.392822 15301 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:25:22.392954 15301 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:25:22.393141 15301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:25:22.397300 15301 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:22.397833 15301 leader_election.cc:302] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5a6a890e2ad47479882dd07e1071ca7; no voters:
I20260807 04:25:22.398480 15301 leader_election.cc:288] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:25:22.398617 15306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:25:22.399787 15306 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [term 2 LEADER]: Becoming Leader. State: Replica: b5a6a890e2ad47479882dd07e1071ca7, State: Running, Role: LEADER
I20260807 04:25:22.400462 15306 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } }
I20260807 04:25:22.400815 15301 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:22.403115 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5a6a890e2ad47479882dd07e1071ca7. Latest consensus state: current_term: 2 leader_uuid: "b5a6a890e2ad47479882dd07e1071ca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } } }
I20260807 04:25:22.403549 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:22.404207 15307 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b5a6a890e2ad47479882dd07e1071ca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a6a890e2ad47479882dd07e1071ca7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40123 } } }
I20260807 04:25:22.404552 15307 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:22.412909 15312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:22.420295 15312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:22.422662 15312 catalog_manager.cc:1259] Loaded cluster ID: cc1fe4ac76d4499aad8c793b1c27f1e0
I20260807 04:25:22.422859 15312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:25:22.428580 15312 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:25:22.430656 15312 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Loaded TSK: 0
I20260807 04:25:22.432857 15312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:25:22.436458 15301 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 04:25:22.810994 15303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:22.811465 15303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:22.811661 15303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:22.823594 15303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:22.823849 15303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:22.824137 15303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:25:22.840026 15303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:36577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=46213
--tserver_master_addrs=127.1.244.254:40123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:22.842029 15303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:22.843868 15303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:22.893962 15343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:22.895598 15342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:22.900616 15345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:22.902257 15303 server_base.cc:1034] running on GCE node
I20260807 04:25:22.903122 15303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:22.904568 15303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:22.905880 15303 hybrid_clock.cc:648] HybridClock initialized: now 1786076722905797 us; error 59 us; skew 500 ppm
I20260807 04:25:22.906325 15303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:22 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076722, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:22.915263 15303 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:25:22.928268 15303 webserver.cc:460] Webserver started at http://127.1.244.193:46213/ using document root <none> and password file <none>
I20260807 04:25:22.929348 15303 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:22.929544 15303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:25:22 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076722, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:22.978029 15303 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.003s
I20260807 04:25:22.982012 15355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:22.983814 15303 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 04:25:22.984038 15303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "d006b1d2635a4315a18ca9fe8948258d"
format_stamp: "Formatted at 2026-08-07 04:25:19 on dist-test-slave-zkpd"
server_key: "169df0bc946ad3ca74e7088b6f55427b"
server_key_iv: "4d12c98f3766ff5bf25f3191016e5290"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:22.985517 15303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:23.037190 15303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:23.041481 15303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:23.041774 15303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:23.042932 15303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:23.044620 15303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:23.044730 15303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:23.044844 15303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:23.044920 15303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:23.099064 15303 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36577
I20260807 04:25:23.099160 15471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36577 every 8 connection(s)
I20260807 04:25:23.101004 15303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:25:23.108534 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15303
I20260807 04:25:23.109409 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:45501
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:40123
--webserver_port=44499
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:25:23 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076722, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:23.169080 15475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:23.125857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40123 (local address 127.1.244.193:49019)
0807 04:25:23.140573 (+ 14716us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:23.140625 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:23.143467 (+ 2842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:23.145518 (+ 2051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:23.145529 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:23.146158 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:23.146963 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:23.146991 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.149649 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.149657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:23.152039 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:23.152059 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.153630 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.153640 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:23.154027 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:23.156115 (+ 2088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:23.156156 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:23.158828 (+ 2672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:23.164054 (+ 5226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:23.164063 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:23.164080 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:23.164647 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:23.165170 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:23.165175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:23.165179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:23.165397 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:23.166424 (+ 1027us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:23.166432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:23.166821 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 04:25:23.167100 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 04:25:23.167636 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12081,"thread_start_us":12196,"threads_started":1}
I20260807 04:25:23.171608 15474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:23.125621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:49019 (local address 127.1.244.254:40123)
0807 04:25:23.138521 (+ 12900us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:23.138545 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:23.140791 (+ 2246us) server_negotiation.cc:407] Connection header received
0807 04:25:23.144316 (+ 3525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:23.144340 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:23.144859 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:23.145303 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:23.147223 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.149439 (+ 2216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.152234 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.153307 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.153507 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:23.159199 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:23.159213 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:23.159236 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:23.159294 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:23.163854 (+ 4560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:23.164828 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:23.164837 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:23.164844 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:23.165036 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:23.165521 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:23.165526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:23.165529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:23.166145 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:23.166340 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:23.169918 (+ 3578us) server_negotiation.cc:299] Negotiation successful
0807 04:25:23.170347 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12577,"thread_start_us":12778,"threads_started":1}
I20260807 04:25:23.174839 15472 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40123
I20260807 04:25:23.175176 15472 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:23.176116 15472 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
W20260807 04:25:23.179641 15472 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:40123 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 04:25:23.179924 15472 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:23.180213 15472 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:23.181595 15472 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:23.181963 15472 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
W20260807 04:25:23.183162 15472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:25:23.538985 15476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:23.539433 15476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:23.539614 15476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:23.549995 15476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:23.550211 15476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:23.550444 15476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:25:23.566303 15476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:45501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=44499
--tserver_master_addrs=127.1.244.254:40123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:23.568145 15476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:23.569905 15476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:23.586155 15492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:23.586974 15493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:23.600172 15476 server_base.cc:1034] running on GCE node
W20260807 04:25:23.601091 15495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:23.602087 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:23.603448 15476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:23.604740 15476 hybrid_clock.cc:648] HybridClock initialized: now 1786076723604672 us; error 57 us; skew 500 ppm
I20260807 04:25:23.605165 15476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:23 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076723, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:23.609659 15476 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:25:23.611692 15476 webserver.cc:460] Webserver started at http://127.1.244.194:44499/ using document root <none> and password file <none>
I20260807 04:25:23.612732 15476 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:23.612887 15476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:25:23 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076723, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
Aug 07 04:25:23 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786076723, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:23.649573 15476 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.010s sys 0.000s
I20260807 04:25:23.658445 15504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:23.660493 15476 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.009s sys 0.001s
I20260807 04:25:23.660789 15476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "7595fd5f5c824b36bd514f7a567cd297"
format_stamp: "Formatted at 2026-08-07 04:25:20 on dist-test-slave-zkpd"
server_key: "5edd4f6b7935c8d72776990dcbe1cb81"
server_key_iv: "8075a3e172ece89e21da6918f0fd868f"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:23.662374 15476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:23.703272 15476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:23.707316 15476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:23.707610 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:23.708745 15476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:23.710505 15476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:23.710623 15476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:23.710754 15476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:23.710839 15476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:23.758560 15476 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45501
I20260807 04:25:23.758664 15625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45501 every 8 connection(s)
I20260807 04:25:23.760068 15476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:25:23.764075 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15476
I20260807 04:25:23.765107 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.244.212:58923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:37895
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:40123
--webserver_port=39111
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:25:23 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076723, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:23.826210 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:23.763013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:42917 (local address 127.1.244.254:40123)
0807 04:25:23.791326 (+ 28313us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:23.791335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:23.794407 (+ 3072us) server_negotiation.cc:407] Connection header received
0807 04:25:23.795548 (+ 1141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:23.795557 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:23.795668 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:23.795778 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:23.798253 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.799506 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.802397 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.803161 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.803294 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:23.811965 (+ 8671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:23.811973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:23.811978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:23.812021 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:23.815533 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:23.820659 (+ 5126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:23.820666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:23.820670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:23.820818 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:23.821347 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:23.821353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:23.821356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:23.821730 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:23.821859 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:23.825539 (+ 3680us) server_negotiation.cc:299] Negotiation successful
0807 04:25:23.825800 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28089,"thread_start_us":27955,"threads_started":1}
I20260807 04:25:23.828706 15629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:23.763389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40123 (local address 127.1.244.194:42917)
0807 04:25:23.792006 (+ 28617us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:23.792076 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:23.795188 (+ 3112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:23.796397 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:23.796412 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:23.797143 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:23.798036 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:23.798070 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.799733 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.799740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:23.802208 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:23.802238 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:23.803709 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:23.803720 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:23.804108 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:23.805814 (+ 1706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:23.805852 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:23.811655 (+ 5803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:23.819829 (+ 8174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:23.819840 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:23.819860 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:23.820512 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:23.820980 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:23.820986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:23.820990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:23.821221 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:23.824555 (+ 3334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:23.824565 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:23.825021 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 04:25:23.826450 (+ 1429us) client_negotiation.cc:240] Negotiation successful
0807 04:25:23.827022 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28241,"thread_start_us":28000,"threads_started":1}
I20260807 04:25:23.831233 15626 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40123
I20260807 04:25:23.831583 15626 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:23.832569 15626 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:23.834679 15264 ts_manager.cc:194] Registered new tserver with Master: 7595fd5f5c824b36bd514f7a567cd297 (127.1.244.194:45501)
I20260807 04:25:23.836828 15264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:42917
Aug 07 04:25:24 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076723, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:24.183305 15472 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:24.183749 15472 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:24.185108 15264 ts_manager.cc:194] Registered new tserver with Master: d006b1d2635a4315a18ca9fe8948258d (127.1.244.193:36577)
I20260807 04:25:24.186293 15264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:49019
W20260807 04:25:24.248037 15630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:24.248425 15630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:24.248591 15630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:24.260077 15630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:24.260213 15630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:24.260438 15630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:25:24.276237 15630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.244.212:58923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:37895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=39111
--tserver_master_addrs=127.1.244.254:40123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:24.278039 15630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:24.279927 15630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:24.296615 15635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:24.297818 15636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:24.306049 15638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:24.306540 15630 server_base.cc:1034] running on GCE node
I20260807 04:25:24.307305 15630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:24.308543 15630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:24.309798 15630 hybrid_clock.cc:648] HybridClock initialized: now 1786076724309732 us; error 55 us; skew 500 ppm
I20260807 04:25:24.310218 15630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:24 dist-test-slave-zkpd krb5kdc[14203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076724, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:24.314414 15630 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:25:24.316397 15630 webserver.cc:460] Webserver started at http://127.1.244.195:39111/ using document root <none> and password file <none>
I20260807 04:25:24.317322 15630 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:24.317497 15630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 04:25:24 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076724, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:24.339975 15630 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 04:25:24.352746 15647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:24.354547 15630 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.004s
I20260807 04:25:24.354717 15630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "f0cad0ae534343379d439df05c60f468"
format_stamp: "Formatted at 2026-08-07 04:25:21 on dist-test-slave-zkpd"
server_key: "6d929c9645ab39483820f48f7ef66c2f"
server_key_iv: "0b8655be58b33f42bf75227941741e84"
server_key_version: "kuduclusterkey@0"
I20260807 04:25:24.356034 15630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:24.387342 15630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:24.392540 15630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:24.392884 15630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:24.396306 15630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:24.411656 15630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:24.411787 15630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:24.411926 15630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:24.412009 15630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 04:25:24 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076723, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for HTTP/127.1.244.212@KRBTEST.COM
I20260807 04:25:24.532303 15630 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:37895
I20260807 04:25:24.532346 15767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:37895 every 8 connection(s)
I20260807 04:25:24.534301 15630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:25:24.538849 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15630
Aug 07 04:25:24 dist-test-slave-zkpd krb5kdc[14203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076724, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:24.570518 15772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.540207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:35275 (local address 127.1.244.254:40123)
0807 04:25:24.544430 (+ 4223us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.544440 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.545736 (+ 1296us) server_negotiation.cc:407] Connection header received
0807 04:25:24.548077 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.548083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.548201 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.548328 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:24.550113 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.551045 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.553263 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.553867 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.554006 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.559180 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:24.559187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:24.559212 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:24.559276 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:24.564123 (+ 4847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:24.566271 (+ 2148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:24.566277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:24.566280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:24.566422 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:24.567104 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:24.567109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:24.567112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:24.567447 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:24.567585 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.569785 (+ 2200us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.570037 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4036,"thread_start_us":3887,"threads_started":1}
I20260807 04:25:24.573530 15770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.539504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40123 (local address 127.1.244.195:35275)
0807 04:25:24.545578 (+ 6074us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:24.545617 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:24.547653 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:24.548814 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:24.548842 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:24.549372 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:24.549939 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.549957 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.551260 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.551264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.553046 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.553071 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.554092 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.554100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.554467 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.555368 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:24.555408 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:24.558908 (+ 3500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:24.565411 (+ 6503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:24.565420 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:24.565441 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:24.566127 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:24.566562 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:24.566567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:24.566571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:24.566964 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:24.568043 (+ 1079us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:24.568052 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:24.568505 (+ 453us) client_negotiation.cc:769] Sending connection context
0807 04:25:24.571110 (+ 2605us) client_negotiation.cc:240] Negotiation successful
0807 04:25:24.572007 (+ 897us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5820,"thread_start_us":5614,"threads_started":1}
I20260807 04:25:24.576467 15768 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40123
I20260807 04:25:24.576807 15768 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:24.577672 15768 heartbeater.cc:511] Master 127.1.244.254:40123 requested a full tablet report, sending...
I20260807 04:25:24.579493 15264 ts_manager.cc:194] Registered new tserver with Master: f0cad0ae534343379d439df05c60f468 (127.1.244.195:37895)
I20260807 04:25:24.580631 15264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:35275
I20260807 04:25:24.591015 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.544301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.1.244.254:40123)
0807 04:25:24.559173 (+ 14872us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.559178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.559194 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:25:24.559240 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.559247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.559338 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.559459 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:24.561871 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.565215 (+ 3344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.567635 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.569317 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.569450 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.574001 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:24.574008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:24.574013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:24.574054 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:24.583725 (+ 9671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:24.587739 (+ 4014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:24.587746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:24.587750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:24.587939 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:24.588463 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:24.588468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:24.588471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:24.588824 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:24.590384 (+ 1560us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.590489 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.590655 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14742,"thread_start_us":15344,"threads_started":1}
I20260807 04:25:24.594120 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:25:24.632649 15773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.602727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59900 (local address 127.1.244.254:40123)
0807 04:25:24.603049 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.603057 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.613263 (+ 10206us) server_negotiation.cc:407] Connection header received
0807 04:25:24.613828 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.613835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.613934 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.614064 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:24.616395 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.617645 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.620000 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.620768 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.620880 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.625506 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:24.625513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:24.625518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:24.625560 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:24.629307 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:24.630182 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:24.630188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:24.630191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:24.630341 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:24.630805 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:24.630810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:24.630813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:24.631164 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:24.631404 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.632031 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.632290 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 04:25:24.639830 15264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:25:24.651530 15264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:25:24.696959 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.675494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:37895 (local address 127.0.0.1:55796)
0807 04:25:24.685104 (+ 9610us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:24.685123 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:24.685254 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:24.690241 (+ 4987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:24.690245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:24.690269 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:24.690762 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.690776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.692894 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.692900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.694337 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.694349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.695926 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.695933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.696135 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.696198 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 04:25:24.696314 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 04:25:24.696510 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9469,"thread_start_us":7803,"threads_started":1}
I20260807 04:25:24.698957 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.676295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.1.244.195:37895)
0807 04:25:24.689720 (+ 13425us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.689727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.689761 (+ 34us) server_negotiation.cc:407] Connection header received
0807 04:25:24.689853 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.689859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.690065 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.690255 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:24.690972 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.692762 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.694482 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.695627 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.697960 (+ 2333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.698192 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.698267 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.698415 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13240,"thread_start_us":13075,"threads_started":1}
I20260807 04:25:24.702720 15787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.674426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:45501 (local address 127.0.0.1:40316)
0807 04:25:24.683848 (+ 9422us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:24.683920 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:24.684146 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:24.686428 (+ 2282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:24.686436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:24.686465 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:24.686946 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.686963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.689483 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.689490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.691686 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.691702 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.701782 (+ 10080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.701789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.701961 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.701996 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:25:24.702091 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 04:25:24.702237 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9040,"mutex_wait_us":3,"thread_start_us":10390,"threads_started":1}
I20260807 04:25:24.703382 15701 tablet_service.cc:1467] Processing CreateTablet for tablet 974d049b476440c892b7b703a7194271 (DEFAULT_TABLE table=test-table [id=dec9078492304cfb941617d660e4e4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:24.705271 15701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974d049b476440c892b7b703a7194271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:24.705526 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.674951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.1.244.194:45501)
0807 04:25:24.685451 (+ 10500us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.685461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.685492 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:25:24.685592 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.685600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.685876 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.686086 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:24.687268 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.689294 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.700076 (+ 10782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.701574 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.701725 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.701928 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.705003 (+ 3075us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.705188 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9631,"thread_start_us":9496,"threads_started":1}
I20260807 04:25:24.705559 15790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.686121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36577 (local address 127.0.0.1:53734)
0807 04:25:24.688820 (+ 2699us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:24.688839 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:24.688948 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:24.689879 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:24.689884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:24.689908 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:24.696848 (+ 6940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.696863 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.698812 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.698817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.702562 (+ 3745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.702578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.704775 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.704784 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.704955 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.705004 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 04:25:24.705090 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 04:25:24.705234 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2576,"thread_start_us":2958,"threads_started":1}
I20260807 04:25:24.706344 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.685668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.1.244.193:36577)
0807 04:25:24.688394 (+ 2726us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.688402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.689065 (+ 663us) server_negotiation.cc:407] Connection header received
0807 04:25:24.689185 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.689193 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.689440 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.689625 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:24.697106 (+ 7481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.698615 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.702772 (+ 4157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.704523 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.704659 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.704881 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.705826 (+ 945us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.705990 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2560,"thread_start_us":2354,"threads_started":1}
I20260807 04:25:24.709425 15558 tablet_service.cc:1467] Processing CreateTablet for tablet 974d049b476440c892b7b703a7194271 (DEFAULT_TABLE table=test-table [id=dec9078492304cfb941617d660e4e4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:24.709996 15403 tablet_service.cc:1467] Processing CreateTablet for tablet 974d049b476440c892b7b703a7194271 (DEFAULT_TABLE table=test-table [id=dec9078492304cfb941617d660e4e4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:24.710831 15558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974d049b476440c892b7b703a7194271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:24.711320 15403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974d049b476440c892b7b703a7194271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:24.728668 15799 tablet_bootstrap.cc:492] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Bootstrap starting.
I20260807 04:25:24.734112 15799 tablet_bootstrap.cc:654] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:24.734972 15797 tablet_bootstrap.cc:492] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Bootstrap starting.
I20260807 04:25:24.735774 15798 tablet_bootstrap.cc:492] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Bootstrap starting.
I20260807 04:25:24.737263 15799 log.cc:824] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:24.740177 15798 tablet_bootstrap.cc:654] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:24.740935 15799 tablet_bootstrap.cc:492] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: No bootstrap required, opened a new log
I20260807 04:25:24.741204 15798 log.cc:824] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:24.741212 15799 ts_tablet_manager.cc:1397] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260807 04:25:24.741587 15797 tablet_bootstrap.cc:654] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:24.742846 15797 log.cc:824] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:24.745476 15797 tablet_bootstrap.cc:492] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: No bootstrap required, opened a new log
I20260807 04:25:24.745754 15797 ts_tablet_manager.cc:1397] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 04:25:24.745770 15798 tablet_bootstrap.cc:492] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: No bootstrap required, opened a new log
I20260807 04:25:24.746018 15798 ts_tablet_manager.cc:1397] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260807 04:25:24.751873 15799 raft_consensus.cc:348] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.752328 15799 raft_consensus.cc:374] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:24.752434 15799 raft_consensus.cc:729] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d006b1d2635a4315a18ca9fe8948258d, State: Initialized, Role: FOLLOWER
I20260807 04:25:24.753058 15799 consensus_queue.cc:260] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.753520 15797 raft_consensus.cc:348] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.753866 15797 raft_consensus.cc:374] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:24.754015 15797 raft_consensus.cc:729] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7595fd5f5c824b36bd514f7a567cd297, State: Initialized, Role: FOLLOWER
I20260807 04:25:24.754552 15472 heartbeater.cc:503] Master 127.1.244.254:40123 was elected leader, sending a full tablet report...
I20260807 04:25:24.754638 15797 consensus_queue.cc:260] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.755329 15799 ts_tablet_manager.cc:1428] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260807 04:25:24.756441 15798 raft_consensus.cc:348] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.756851 15798 raft_consensus.cc:374] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:24.756896 15626 heartbeater.cc:503] Master 127.1.244.254:40123 was elected leader, sending a full tablet report...
I20260807 04:25:24.756990 15798 raft_consensus.cc:729] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0cad0ae534343379d439df05c60f468, State: Initialized, Role: FOLLOWER
I20260807 04:25:24.757486 15797 ts_tablet_manager.cc:1428] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260807 04:25:24.757651 15798 consensus_queue.cc:260] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
W20260807 04:25:24.762357 15627 tablet.cc:2369] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:24.764791 15768 heartbeater.cc:503] Master 127.1.244.254:40123 was elected leader, sending a full tablet report...
I20260807 04:25:24.765630 15798 ts_tablet_manager.cc:1428] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Time spent starting tablet: real 0.019s user 0.013s sys 0.007s
W20260807 04:25:24.793273 15769 tablet.cc:2369] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:25:24.881268 15473 tablet.cc:2369] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:24.935518 15804 raft_consensus.cc:482] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:25:24.935832 15804 raft_consensus.cc:504] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.937561 15804 leader_election.cc:288] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0cad0ae534343379d439df05c60f468 (127.1.244.195:37895), d006b1d2635a4315a18ca9fe8948258d (127.1.244.193:36577)
I20260807 04:25:24.944308 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.938112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35405 (local address 127.1.244.193:36577)
0807 04:25:24.938331 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.938337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.938973 (+ 636us) server_negotiation.cc:407] Connection header received
0807 04:25:24.939179 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.939184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.939240 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.939330 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:24.939884 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.940628 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.942490 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.943283 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.943408 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.943459 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.943852 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.944037 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:25:24.944342 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.937856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:39833 (local address 127.1.244.195:37895)
0807 04:25:24.938047 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.938056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.938992 (+ 936us) server_negotiation.cc:407] Connection header received
0807 04:25:24.939231 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.939235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.939294 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.939384 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:24.940068 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.940851 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.942638 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.943433 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.943595 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.943668 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.943977 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.944122 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:25:24.944411 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.937816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36577 (local address 127.1.244.194:35405)
0807 04:25:24.938871 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:24.938885 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:24.939040 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:24.939347 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:24.939351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:24.939383 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:24.939742 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.939752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.940801 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.940806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.942339 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.942351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.943487 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.943496 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.943681 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.943722 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 04:25:24.943794 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 04:25:24.943961 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":4,"thread_start_us":590,"threads_started":1}
I20260807 04:25:24.944481 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.937930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:37895 (local address 127.1.244.194:39833)
0807 04:25:24.938868 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:24.938887 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:24.939086 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:24.939397 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:24.939401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:24.939423 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:24.939837 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.939847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.940975 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.940979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.942487 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:24.942501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.943631 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.943636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:24.943754 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.943799 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 04:25:24.943925 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 04:25:24.944042 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"spinlock_wait_cycles":24320,"thread_start_us":702,"threads_started":1}
I20260807 04:25:24.945451 15721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974d049b476440c892b7b703a7194271" candidate_uuid: "7595fd5f5c824b36bd514f7a567cd297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0cad0ae534343379d439df05c60f468" is_pre_election: true
I20260807 04:25:24.945521 15425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974d049b476440c892b7b703a7194271" candidate_uuid: "7595fd5f5c824b36bd514f7a567cd297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d006b1d2635a4315a18ca9fe8948258d" is_pre_election: true
I20260807 04:25:24.945941 15721 raft_consensus.cc:2454] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7595fd5f5c824b36bd514f7a567cd297 in term 0.
I20260807 04:25:24.946012 15425 raft_consensus.cc:2454] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7595fd5f5c824b36bd514f7a567cd297 in term 0.
I20260807 04:25:24.946569 15508 leader_election.cc:302] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7595fd5f5c824b36bd514f7a567cd297, f0cad0ae534343379d439df05c60f468; no voters:
I20260807 04:25:24.946940 15804 raft_consensus.cc:2781] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:25:24.947081 15804 raft_consensus.cc:482] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:25:24.947180 15804 raft_consensus.cc:3037] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:24.949448 15804 raft_consensus.cc:504] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.950042 15804 leader_election.cc:288] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [CANDIDATE]: Term 1 election: Requested vote from peers f0cad0ae534343379d439df05c60f468 (127.1.244.195:37895), d006b1d2635a4315a18ca9fe8948258d (127.1.244.193:36577)
I20260807 04:25:24.950467 15721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974d049b476440c892b7b703a7194271" candidate_uuid: "7595fd5f5c824b36bd514f7a567cd297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0cad0ae534343379d439df05c60f468"
I20260807 04:25:24.950476 15425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974d049b476440c892b7b703a7194271" candidate_uuid: "7595fd5f5c824b36bd514f7a567cd297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d006b1d2635a4315a18ca9fe8948258d"
I20260807 04:25:24.950659 15425 raft_consensus.cc:3037] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:24.950663 15721 raft_consensus.cc:3037] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:24.953859 15425 raft_consensus.cc:2454] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7595fd5f5c824b36bd514f7a567cd297 in term 1.
I20260807 04:25:24.953859 15721 raft_consensus.cc:2454] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7595fd5f5c824b36bd514f7a567cd297 in term 1.
I20260807 04:25:24.954264 15508 leader_election.cc:302] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7595fd5f5c824b36bd514f7a567cd297, d006b1d2635a4315a18ca9fe8948258d, f0cad0ae534343379d439df05c60f468; no voters:
I20260807 04:25:24.954576 15804 raft_consensus.cc:2781] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:24.955032 15804 raft_consensus.cc:686] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 1 LEADER]: Becoming Leader. State: Replica: 7595fd5f5c824b36bd514f7a567cd297, State: Running, Role: LEADER
I20260807 04:25:24.955546 15804 consensus_queue.cc:237] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } }
I20260807 04:25:24.959872 15263 catalog_manager.cc:5495] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7595fd5f5c824b36bd514f7a567cd297 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "7595fd5f5c824b36bd514f7a567cd297" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7595fd5f5c824b36bd514f7a567cd297" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 } health_report { overall_health: UNKNOWN } } }
I20260807 04:25:24.990957 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:24.986082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.1.244.194:45501)
0807 04:25:24.986250 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:24.986255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:24.986272 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:25:24.986411 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:24.986416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:24.986510 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:24.986630 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:25:24.986948 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.987661 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.989008 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:24.989391 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:24.989485 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:24.989888 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:25:24.990387 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:25:24.990486 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:24.990621 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 04:25:24.990719 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 04:25:24.998867 15425 raft_consensus.cc:1261] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 1 FOLLOWER]: Refusing update from remote peer 7595fd5f5c824b36bd514f7a567cd297: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:24.998867 15721 raft_consensus.cc:1261] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 1 FOLLOWER]: Refusing update from remote peer 7595fd5f5c824b36bd514f7a567cd297: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:24.999923 15808 consensus_queue.cc:1035] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0cad0ae534343379d439df05c60f468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:25.000289 15804 consensus_queue.cc:1035] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d006b1d2635a4315a18ca9fe8948258d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:25.016943 15811 mvcc.cc:205] Tried to move back new op lower bound from 7315770265585782784 to 7315770265421774848. Current Snapshot: MvccSnapshot[applied={T|T < 7315770265585782784}]
I20260807 04:25:25.033169 15263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:25:25.033468 15263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:25:25.037321 15263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5a6a890e2ad47479882dd07e1071ca7: Sending DeleteTablet for 3 replicas of tablet 974d049b476440c892b7b703a7194271
I20260807 04:25:25.038498 15558 tablet_service.cc:1514] Processing DeleteTablet for tablet 974d049b476440c892b7b703a7194271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260807 04:25:25.038518 15701 tablet_service.cc:1514] Processing DeleteTablet for tablet 974d049b476440c892b7b703a7194271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:55796
I20260807 04:25:25.038614 15403 tablet_service.cc:1514] Processing DeleteTablet for tablet 974d049b476440c892b7b703a7194271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:53734
I20260807 04:25:25.049468 15821 tablet_replica.cc:330] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: stopping tablet replica
I20260807 04:25:25.049737 15822 tablet_replica.cc:330] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: stopping tablet replica
I20260807 04:25:25.049929 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15303
I20260807 04:25:25.050227 15822 raft_consensus.cc:2229] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:25:25.050534 15821 raft_consensus.cc:2229] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:25:25.050920 15822 raft_consensus.cc:2258] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:25.051343 15820 tablet_replica.cc:330] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: stopping tablet replica
I20260807 04:25:25.051342 15821 raft_consensus.cc:2258] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:25.051764 15820 raft_consensus.cc:2229] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:25:25.052271 15820 raft_consensus.cc:2258] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:25.053517 15822 ts_tablet_manager.cc:1905] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:25.053524 15821 ts_tablet_manager.cc:1905] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:25.055256 15820 ts_tablet_manager.cc:1905] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:25.060811 15822 ts_tablet_manager.cc:1918] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:25.061075 15822 log.cc:1196] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/974d049b476440c892b7b703a7194271
I20260807 04:25:25.061687 15822 ts_tablet_manager.cc:1939] T 974d049b476440c892b7b703a7194271 P d006b1d2635a4315a18ca9fe8948258d: Deleting consensus metadata
I20260807 04:25:25.063548 15250 catalog_manager.cc:4841] TS d006b1d2635a4315a18ca9fe8948258d (127.1.244.193:36577): tablet 974d049b476440c892b7b703a7194271 (table test-table [id=dec9078492304cfb941617d660e4e4ed]) successfully deleted
I20260807 04:25:25.066875 15820 ts_tablet_manager.cc:1918] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:25.067055 15821 ts_tablet_manager.cc:1918] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:25.067216 15820 log.cc:1196] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/974d049b476440c892b7b703a7194271
I20260807 04:25:25.067216 15821 log.cc:1196] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/974d049b476440c892b7b703a7194271
I20260807 04:25:25.067783 15821 ts_tablet_manager.cc:1939] T 974d049b476440c892b7b703a7194271 P 7595fd5f5c824b36bd514f7a567cd297: Deleting consensus metadata
I20260807 04:25:25.067932 15820 ts_tablet_manager.cc:1939] T 974d049b476440c892b7b703a7194271 P f0cad0ae534343379d439df05c60f468: Deleting consensus metadata
I20260807 04:25:25.069629 15251 catalog_manager.cc:4841] TS f0cad0ae534343379d439df05c60f468 (127.1.244.195:37895): tablet 974d049b476440c892b7b703a7194271 (table test-table [id=dec9078492304cfb941617d660e4e4ed]) successfully deleted
I20260807 04:25:25.069819 15250 catalog_manager.cc:4841] TS 7595fd5f5c824b36bd514f7a567cd297 (127.1.244.194:45501): tablet 974d049b476440c892b7b703a7194271 (table test-table [id=dec9078492304cfb941617d660e4e4ed]) successfully deleted
I20260807 04:25:25.071326 15789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:25.051084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53748 (local address 127.1.244.193:36577)
0807 04:25:25.052144 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:25.052150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:25.052168 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:25:25.052237 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:25.052244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:25.052338 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:25.052445 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:25.054525 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:25.055688 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:25.058847 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:25.059547 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:25.059672 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:25.063275 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:25.063290 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:25.063306 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:25.063353 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:25.067060 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:25.068023 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:25.068034 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:25.068040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:25.068238 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:25.068736 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:25.068741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:25.068745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:25.069220 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:25.069538 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:25.070711 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0807 04:25:25.070921 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940}
I20260807 04:25:25.072022 15466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:25.193198 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15303
I20260807 04:25:25.212644 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15476
I20260807 04:25:25.226997 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:25.213495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.1.244.194:45501)
0807 04:25:25.213680 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:25.213697 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:25.213755 (+ 58us) server_negotiation.cc:407] Connection header received
0807 04:25:25.213980 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:25.213985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:25.214070 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:25.214188 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:25.215484 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:25.216447 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:25.217696 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:25.218156 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:25.218311 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:25.220936 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:25.220950 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:25.220965 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:25.221000 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:25.224041 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:25.224674 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:25.224684 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:25.224687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:25.224839 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:25.225217 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:25.225223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:25.225226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:25.225612 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:25.225971 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:25.226364 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 04:25:25.226612 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:25:25.227742 15621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:25.342258 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15476
I20260807 04:25:25.364075 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15630
I20260807 04:25:25.376654 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:25.364783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55798 (local address 127.1.244.195:37895)
0807 04:25:25.364942 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:25.364946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:25.364980 (+ 34us) server_negotiation.cc:407] Connection header received
0807 04:25:25.365111 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:25.365115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:25.365176 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:25.365249 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:25.366502 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:25.367187 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:25.368497 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:25.368868 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:25.368966 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:25.371440 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:25.371455 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:25.371469 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:25.371517 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:25.374348 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:25.374911 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:25.374917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:25.374920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:25.375043 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:25.375342 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:25.375345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:25.375347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:25.375670 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:25.376139 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:25.376251 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 04:25:25.376399 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 04:25:25.377251 15763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:25.467432 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15630
I20260807 04:25:25.485147 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15231
I20260807 04:25:25.486300 15294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:25.599730 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15231
I20260807 04:25:25.623725 2003 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 04:25:25.968637 2003 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 04:25:25.968902 2003 mini_ranger.cc:66] Stopping Ranger...
I20260807 04:25:26.314661 2003 mini_ranger.cc:68] Stopped Ranger
2026-08-07 04:25:26.314 UTC [14234] LOG: received smart shutdown request
2026-08-07 04:25:26.317 UTC [14234] LOG: background worker "logical replication launcher" (PID 14243) exited with exit code 1
2026-08-07 04:25:26.318 UTC [14238] LOG: shutting down
2026-08-07 04:25:26.470 UTC [14234] LOG: database system is shut down
2026-08-07T04:25:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52698 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:25:26 dist-test-slave-zkpd krb5kdc[15837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:25:26 dist-test-slave-zkpd krb5kdc[15837](info): set up 2 sockets
Aug 07 04:25:26 dist-test-slave-zkpd krb5kdc[15837](info): commencing operation
krb5kdc: starting...
W20260807 04:25:27.727285 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:25:27 dist-test-slave-zkpd krb5kdc[15837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:25:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:25:27.995183 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36495
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36495,127.1.244.253:42845,127.1.244.252:45959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:28.332705 15851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:28.333067 15851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:28.333235 15851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:28.340601 15851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:28.340755 15851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:28.340868 15851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:28.340974 15851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:28.341059 15851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:28.350926 15851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36495,127.1.244.253:42845,127.1.244.252:45959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:28.352375 15851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:28.353731 15851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:28.365625 15857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:28.365659 15859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:28.365728 15856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:28.369297 15851 server_base.cc:1034] running on GCE node
I20260807 04:25:28.370095 15851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:28.371531 15851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:28.372759 15851 hybrid_clock.cc:648] HybridClock initialized: now 1786076728372720 us; error 47 us; skew 500 ppm
I20260807 04:25:28.373111 15851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:28 dist-test-slave-zkpd krb5kdc[15837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:28.380638 15851 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:28.382320 15851 webserver.cc:460] Webserver started at http://127.1.244.254:41841/ using document root <none> and password file <none>
I20260807 04:25:28.383065 15851 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:28.383239 15851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:28.383551 15851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:28.386667 15851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "315f1cbdf6cd43c0a001317864341e73"
format_stamp: "Formatted at 2026-08-07 04:25:28 on dist-test-slave-zkpd"
server_key: "4ad7761c1f11cf23fdc31b149aec0144"
server_key_iv: "8dbadbc2d86abe42f9ad22ef1b50eeb4"
server_key_version: "encryptionkey@0"
I20260807 04:25:28.387494 15851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "315f1cbdf6cd43c0a001317864341e73"
format_stamp: "Formatted at 2026-08-07 04:25:28 on dist-test-slave-zkpd"
server_key: "4ad7761c1f11cf23fdc31b149aec0144"
server_key_iv: "8dbadbc2d86abe42f9ad22ef1b50eeb4"
server_key_version: "encryptionkey@0"
I20260807 04:25:28.392727 15851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:25:28.396106 15866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:28.397393 15851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:25:28.397572 15851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "315f1cbdf6cd43c0a001317864341e73"
format_stamp: "Formatted at 2026-08-07 04:25:28 on dist-test-slave-zkpd"
server_key: "4ad7761c1f11cf23fdc31b149aec0144"
server_key_iv: "8dbadbc2d86abe42f9ad22ef1b50eeb4"
server_key_version: "encryptionkey@0"
I20260807 04:25:28.397766 15851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:28.417151 15851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:28.421320 15851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:28.421622 15851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:28.441200 15851 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:36495
I20260807 04:25:28.441286 15919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:36495 every 8 connection(s)
I20260807 04:25:28.442577 15851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:28.446192 15920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:28.450140 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15851
I20260807 04:25:28.450397 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:25:28.451083 2003 external_mini_cluster.cc:1428] Setting key 60fd5c36353be509d7e9313eb0c62b6e
I20260807 04:25:28.452987 15920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:25:28 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.253@KRBTEST.COM; defaulting to no policy
I20260807 04:25:28.475129 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.456034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.1.244.254:36495)
0807 04:25:28.457853 (+ 1819us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:28.457865 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:28.457915 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:25:28.459596 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:28.459602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:28.459815 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:28.460077 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:28.461359 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.462574 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.463957 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.464515 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.464711 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:28.468127 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:28.468135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:28.468147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:28.468179 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:28.470912 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:28.471582 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:28.471595 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:28.471597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:28.471740 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:28.472181 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:28.472190 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:28.472192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:28.472663 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:28.472855 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:28.473387 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 04:25:28.473691 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":1605,"spinlock_wait_cycles":16768,"thread_start_us":1381,"threads_started":1}
I20260807 04:25:28.475159 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.456033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36495 (local address 127.0.0.1:34918)
0807 04:25:28.457033 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.457088 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:28.459220 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:28.460083 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:28.460090 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:28.460823 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:28.461164 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:28.461182 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.462726 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.462732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:28.463792 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:28.463805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.464750 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.464756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:28.465037 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:28.465689 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:28.465717 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:28.467825 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:28.471044 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:28.471057 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:28.471069 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:28.471451 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:28.471847 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:28.471853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:28.471856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:28.472070 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:28.472795 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:28.472803 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:28.473018 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 04:25:28.473250 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 04:25:28.473804 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":188,"thread_start_us":491,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.253@KRBTEST.COM" created.
I20260807 04:25:28.478379 15920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:25:28.479564 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.479193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52306)
0807 04:25:28.479351 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.479456 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 04:25:28.479839 15870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
I20260807 04:25:28.481365 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.481038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52318)
0807 04:25:28.481185 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.481246 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:25:28.481559 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:42845: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
I20260807 04:25:28.507274 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:42845
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36495,127.1.244.253:42845,127.1.244.252:45959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:28.530925 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } attempt: 1
I20260807 04:25:28.532277 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.531715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52326)
0807 04:25:28.531962 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.532109 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 04:25:28.533286 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.532910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52336)
0807 04:25:28.533055 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.533137 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 04:25:28.533517 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:42845: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
I20260807 04:25:28.601764 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } attempt: 2
I20260807 04:25:28.603083 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.602468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52352)
0807 04:25:28.602766 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.602884 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 04:25:28.604279 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.603737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52356)
0807 04:25:28.603968 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.604094 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 04:25:28.604532 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:42845: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
I20260807 04:25:28.695863 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } attempt: 3
I20260807 04:25:28.697331 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.696687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52366)
0807 04:25:28.696960 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.697124 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 04:25:28.698557 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.698041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52380)
0807 04:25:28.698239 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.698345 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 04:25:28.698832 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:42845: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
W20260807 04:25:28.839044 15927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:28.839304 15927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:28.839397 15927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 04:25:28.842098 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } attempt: 4
I20260807 04:25:28.843343 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.842745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52382)
0807 04:25:28.843062 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.843176 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260807 04:25:28.844451 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.844028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52388)
0807 04:25:28.844195 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.844295 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 04:25:28.844653 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:42845: Network error: Client connection negotiation failed: client connection to 127.1.244.253:42845: connect: Connection refused (error 111)
W20260807 04:25:28.847996 15927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:28.848127 15927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:28.848222 15927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:28.848302 15927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:28.848378 15927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:28.858112 15927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36495,127.1.244.253:42845,127.1.244.252:45959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:28.859409 15927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:28.860828 15927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:28.872390 15932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:28.872308 15935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:28.872352 15933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:28.873387 15927 server_base.cc:1034] running on GCE node
I20260807 04:25:28.874101 15927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:28.875358 15927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:28.876572 15927 hybrid_clock.cc:648] HybridClock initialized: now 1786076728876533 us; error 46 us; skew 500 ppm
I20260807 04:25:28.876897 15927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:28 dist-test-slave-zkpd krb5kdc[15837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:28.879849 15927 init.cc:377] Logged in from keytab as kudu/127.1.244.253@KRBTEST.COM (short username kudu)
I20260807 04:25:28.881237 15927 webserver.cc:460] Webserver started at http://127.1.244.253:32823/ using document root <none> and password file <none>
I20260807 04:25:28.881847 15927 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:28.881942 15927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:28.882179 15927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:28.885040 15927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data/instance:
uuid: "5366a4e646f147909701d4badebc5473"
format_stamp: "Formatted at 2026-08-07 04:25:28 on dist-test-slave-zkpd"
server_key: "d25ba30e46e832f827b1a20d887297e3"
server_key_iv: "7e0d9d1b9049eb80a32732e22fe3cc4c"
server_key_version: "encryptionkey@0"
I20260807 04:25:28.885778 15927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal/instance:
uuid: "5366a4e646f147909701d4badebc5473"
format_stamp: "Formatted at 2026-08-07 04:25:28 on dist-test-slave-zkpd"
server_key: "d25ba30e46e832f827b1a20d887297e3"
server_key_iv: "7e0d9d1b9049eb80a32732e22fe3cc4c"
server_key_version: "encryptionkey@0"
I20260807 04:25:28.890561 15927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:25:28.893885 15942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:28.895161 15927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:25:28.895329 15927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal
uuid: "5366a4e646f147909701d4badebc5473"
format_stamp: "Formatted at 2026-08-07 04:25:28 on dist-test-slave-zkpd"
server_key: "d25ba30e46e832f827b1a20d887297e3"
server_key_iv: "7e0d9d1b9049eb80a32732e22fe3cc4c"
server_key_version: "encryptionkey@0"
I20260807 04:25:28.895509 15927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:28.912453 15927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:28.916270 15927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:28.916558 15927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:28.934602 15927 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:42845
I20260807 04:25:28.934643 15999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:42845 every 8 connection(s)
I20260807 04:25:28.935935 15927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
I20260807 04:25:28.939327 16000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:28.942154 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 15927
I20260807 04:25:28.942337 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-1/wal/instance
I20260807 04:25:28.942904 2003 external_mini_cluster.cc:1428] Setting key f87189246cc218d20d9b8827a258bdc9
I20260807 04:25:28.947983 16000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:25:28 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.252@KRBTEST.COM; defaulting to no policy
I20260807 04:25:28.968320 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.951284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.1.244.254:36495)
0807 04:25:28.951525 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:28.951534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:28.952677 (+ 1143us) server_negotiation.cc:407] Connection header received
0807 04:25:28.954982 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:28.954988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:28.955083 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:28.955191 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:28.956649 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.957324 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.958814 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.959240 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.959369 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:28.962695 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:28.962700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:28.962703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:28.962735 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:28.965394 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:28.966097 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:28.966101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:28.966104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:28.966218 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:28.966638 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:28.966643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:28.966646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:28.966928 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:28.967079 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:28.967750 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 04:25:28.968002 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:25:28.969323 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.951679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36495 (local address 127.0.0.1:34926)
0807 04:25:28.952512 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.952569 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:28.954709 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:28.955415 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:28.955423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:28.955937 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:28.956481 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:28.956498 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.957486 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.957490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:28.958669 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:28.958680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.959468 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.959476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:28.959684 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:28.960369 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:28.960397 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:28.962475 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:28.965531 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:28.965540 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:28.965555 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:28.965969 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:28.966322 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:28.966327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:28.966331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:28.966522 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:28.967059 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:28.967066 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:28.967376 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 04:25:28.967630 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 04:25:28.968136 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":353,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.252@KRBTEST.COM" created.
I20260807 04:25:28.971094 16000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:25:28 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:25:28.987236 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.971947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52404)
0807 04:25:28.972155 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.972182 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:28.972261 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:28.973208 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:28.973219 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:28.973737 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:28.973947 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:28.973961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.975280 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.975288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:28.976833 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:28.976850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.977616 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.977628 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:28.977864 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:28.978426 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:28.978453 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:28.980639 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:28.984963 (+ 4324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:28.984967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:28.984969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:28.985304 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:28.985726 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:28.985729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:28.985731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:28.985857 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:28.986609 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:28.986612 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:28.986741 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 04:25:28.986834 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 04:25:28.987009 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 04:25:28.987784 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.972205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.1.244.253:42845)
0807 04:25:28.972829 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:28.972834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:28.972857 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:25:28.972897 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:28.972902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:28.973055 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:28.973166 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:28.974110 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.975142 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.976964 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:28.977478 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:28.977572 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:28.980857 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:28.980873 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:28.980892 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:28.980935 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:28.984844 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:28.985446 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:28.985458 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:28.985464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:28.985608 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:28.985990 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:28.985997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:28.986003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:28.986500 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:28.986714 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:28.987027 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 04:25:28.987315 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":361,"threads_started":1}
I20260807 04:25:28.990204 16000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:25:28.991545 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.990937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57646)
0807 04:25:28.991172 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.991304 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 04:25:28.991828 15945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:28.993526 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:28.993108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57652)
0807 04:25:28.993275 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:28.993373 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:25:28.993764 16000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
I20260807 04:25:29.003187 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:45959
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36495,127.1.244.253:42845,127.1.244.252:45959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:29.043140 16000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 1
I20260807 04:25:29.044458 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.043865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57664)
0807 04:25:29.044131 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.044257 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 04:25:29.045446 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.045075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57674)
0807 04:25:29.045215 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.045312 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 04:25:29.045644 16000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.113883 16000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 2
I20260807 04:25:29.115064 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.114537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57682)
0807 04:25:29.114754 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.114867 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 04:25:29.116080 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.115684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57692)
0807 04:25:29.115825 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.115942 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 04:25:29.116355 16000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.144045 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } attempt: 5
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:25:29.158376 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.144935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.1.244.253:42845)
0807 04:25:29.145101 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.145106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.145125 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:25:29.145310 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.145314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.145394 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.145478 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.146703 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.147381 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.149095 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.149533 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.149621 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.153216 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.153221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.153224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.153255 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.155863 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.156541 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.156548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.156552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.156673 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.157030 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.157035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.157038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.157309 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.157461 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.157829 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.158051 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:25:29.158376 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.144759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52414)
0807 04:25:29.145009 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.145033 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:29.145178 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:29.145575 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:29.145582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:29.146208 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:29.146538 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.146558 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.147520 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.147526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.148941 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.148957 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.149679 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.149684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.149867 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.150524 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:29.150547 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:29.153029 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:29.156044 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.156050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.156055 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.156427 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.156780 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.156785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.156789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.156925 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.157430 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:29.157435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:29.157593 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 04:25:29.157734 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 04:25:29.158009 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 04:25:29.159451 15920 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:25:29.160486 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.160104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57700)
0807 04:25:29.160266 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.160353 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 04:25:29.161441 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.161114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57710)
0807 04:25:29.161259 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.161331 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:25:29.161664 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.207597 16000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 3
I20260807 04:25:29.208837 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.208280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57720)
0807 04:25:29.208522 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.208642 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 04:25:29.209980 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.209531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57732)
0807 04:25:29.209760 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.209851 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260807 04:25:29.210209 16000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.212896 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 1
I20260807 04:25:29.213809 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.213386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57738)
0807 04:25:29.213553 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.213655 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 04:25:29.214761 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.214461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57744)
0807 04:25:29.214602 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.214663 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 04:25:29.214993 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.283200 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 2
I20260807 04:25:29.284399 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.283853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57760)
0807 04:25:29.284130 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.284230 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 04:25:29.285405 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.285077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57774)
0807 04:25:29.285247 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.285308 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:25:29.285609 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
W20260807 04:25:29.340539 16007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:29.340879 16007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:29.341039 16007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:29.348219 16007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:29.348363 16007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:29.348470 16007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:29.348565 16007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:29.348645 16007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:29.353436 16000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 4
I20260807 04:25:29.354631 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.354065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57790)
0807 04:25:29.354329 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.354443 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 04:25:29.355620 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.355280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57806)
0807 04:25:29.355420 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.355499 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 04:25:29.355850 16000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.359176 16007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36495,127.1.244.253:42845,127.1.244.252:45959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:45959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:29.360455 16007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:29.361696 16007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:29.373138 16014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:29.373189 16013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:29.373234 16016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:29.374228 16007 server_base.cc:1034] running on GCE node
I20260807 04:25:29.374948 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:29.376324 16007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:29.377526 16007 hybrid_clock.cc:648] HybridClock initialized: now 1786076729377490 us; error 45 us; skew 500 ppm
I20260807 04:25:29.377859 16007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076729, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:29.380972 16007 init.cc:377] Logged in from keytab as kudu/127.1.244.252@KRBTEST.COM (short username kudu)
I20260807 04:25:29.382433 16007 webserver.cc:460] Webserver started at http://127.1.244.252:45943/ using document root <none> and password file <none>
I20260807 04:25:29.383116 16007 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:29.383255 16007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:29.383553 16007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:29.386557 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data/instance:
uuid: "0784ce0675ae4f9281dacf24651d6346"
format_stamp: "Formatted at 2026-08-07 04:25:29 on dist-test-slave-zkpd"
server_key: "bd9f2c6d37a8f10e2a53c44097453298"
server_key_iv: "92afbcca8fb72400d4a8cab6015f828b"
server_key_version: "encryptionkey@0"
I20260807 04:25:29.387339 16007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal/instance:
uuid: "0784ce0675ae4f9281dacf24651d6346"
format_stamp: "Formatted at 2026-08-07 04:25:29 on dist-test-slave-zkpd"
server_key: "bd9f2c6d37a8f10e2a53c44097453298"
server_key_iv: "92afbcca8fb72400d4a8cab6015f828b"
server_key_version: "encryptionkey@0"
I20260807 04:25:29.391809 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 3
I20260807 04:25:29.392282 16007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 04:25:29.392896 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.392435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57810)
0807 04:25:29.392664 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.392746 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:25:29.393971 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.393581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57824)
0807 04:25:29.393793 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.393863 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 04:25:29.394177 15920 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45959: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45959: connect: Connection refused (error 111)
I20260807 04:25:29.395815 16024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:29.397138 16007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:25:29.397315 16007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal
uuid: "0784ce0675ae4f9281dacf24651d6346"
format_stamp: "Formatted at 2026-08-07 04:25:29 on dist-test-slave-zkpd"
server_key: "bd9f2c6d37a8f10e2a53c44097453298"
server_key_iv: "92afbcca8fb72400d4a8cab6015f828b"
server_key_version: "encryptionkey@0"
I20260807 04:25:29.397504 16007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:29.448046 16007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:29.453085 16007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:29.453333 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:29.474272 16007 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:45959
I20260807 04:25:29.474294 16077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:45959 every 8 connection(s)
I20260807 04:25:29.475643 16007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
I20260807 04:25:29.477972 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16007
I20260807 04:25:29.478183 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786076526463677-2003-0/minicluster-data/master-2/wal/instance
I20260807 04:25:29.478757 2003 external_mini_cluster.cc:1428] Setting key 97b506471d82db240079ee6abd6f18b2
I20260807 04:25:29.480038 16078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:29.489038 16078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076729, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:29.501788 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.480692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57840 (local address 127.1.244.252:45959)
0807 04:25:29.481461 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.481473 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.481518 (+ 45us) server_negotiation.cc:407] Connection header received
0807 04:25:29.483026 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.483044 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.483392 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.483710 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.485570 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.486867 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.488797 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.489444 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.489582 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.494397 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.494405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.494414 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.494454 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.497280 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.498142 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.498166 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.498175 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.498356 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.498833 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.498837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.498839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.499361 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.499654 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.500170 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.500549 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":356,"threads_started":1}
I20260807 04:25:29.507067 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.491784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:36495 (local address 127.0.0.1:34932)
0807 04:25:29.492803 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.492852 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:29.492995 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:29.493432 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:29.493442 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:29.493998 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:29.494251 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.494260 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.495449 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.495453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.496638 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.496650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.497378 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.497385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.497566 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.498129 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:29.498145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:29.500658 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:29.504329 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.504342 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.504348 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.504827 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.505256 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.505264 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.505270 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.505443 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.505981 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:29.505989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:29.506155 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 04:25:29.506328 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 04:25:29.506668 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":661,"threads_started":1}
I20260807 04:25:29.507126 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.491835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.1.244.254:36495)
0807 04:25:29.492395 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.492402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.492946 (+ 544us) server_negotiation.cc:407] Connection header received
0807 04:25:29.493141 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.493150 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.493233 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.493337 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.494405 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.495287 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.496785 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.497194 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.497333 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.500848 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.500853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.500856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.500886 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.504214 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.504979 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.504984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.504987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.505149 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.505570 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.505575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.505578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.505825 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.506029 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.506482 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.506695 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":313,"threads_started":1}
I20260807 04:25:29.508508 16078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076729, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:25:29.521629 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.504762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.1.244.254:36495)
0807 04:25:29.505556 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.505560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.505576 (+ 16us) server_negotiation.cc:407] Connection header received
0807 04:25:29.505611 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.505613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.505657 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.505763 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.507035 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.508265 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.509956 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.510607 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.510687 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.515812 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.515819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.515824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.515867 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.518737 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.519605 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.519609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.519611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.519783 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.520204 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.520208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.520210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.520576 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.520858 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.521034 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.521186 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":532,"threads_started":1}
I20260807 04:25:29.522939 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.509157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:42845 (local address 127.0.0.1:52420)
0807 04:25:29.509299 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.509318 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:29.509418 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:29.509724 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:29.509728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:29.510205 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:29.510380 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.510403 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.511675 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.511679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.512581 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.512593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.513598 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.513602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.513782 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.514301 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:29.514325 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:29.516946 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:29.520359 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.520364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.520367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.520863 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.521274 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.521279 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.521283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.521446 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.522046 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:29.522051 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:29.522201 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 04:25:29.522336 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 04:25:29.522588 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 04:25:29.522994 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.509265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.1.244.253:42845)
0807 04:25:29.509436 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.509444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.509465 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:25:29.509524 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.509529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.509593 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.509691 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.510529 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.511513 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.512734 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.513463 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.513611 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.517129 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.517134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.517137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.517167 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.520251 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.520982 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.520987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.520990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.521151 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.521587 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.521592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.521595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.521922 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.522088 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.522433 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.522629 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 04:25:29.524164 16078 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076729, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:25:29.537837 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.524960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57856)
0807 04:25:29.525047 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.525059 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:29.525169 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:29.525432 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:29.525436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:29.525851 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:29.526015 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.526027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.527276 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.527279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.528423 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.528440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.529364 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.529368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.529540 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.530045 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:29.530059 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:29.532163 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:29.535186 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.535191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.535195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.535678 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.536086 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.536092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.536096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.536280 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.536869 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:29.536872 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:29.536990 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 04:25:29.537171 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 04:25:29.537471 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":33}
I20260807 04:25:29.538061 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.524808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.1.244.252:45959)
0807 04:25:29.524954 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.524958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.525119 (+ 161us) server_negotiation.cc:407] Connection header received
0807 04:25:29.525261 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.525265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.525313 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.525378 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.526140 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.527147 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.528551 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.529200 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.529350 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.532343 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.532350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.532356 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.532391 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.535061 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.535786 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.535794 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.535796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.535973 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.536396 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.536403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.536406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.536755 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.536920 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.537257 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.537688 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":42}
I20260807 04:25:29.539631 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.523179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.1.244.253:42845)
0807 04:25:29.523351 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.523355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.523384 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:25:29.523516 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.523521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.523585 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.523664 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.525189 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.526262 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.527753 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.528396 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.528535 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.533250 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.533260 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.533264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.533294 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.536311 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.537072 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.537077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.537081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.537237 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.537651 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.537657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.537660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.537997 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.538286 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.539155 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.539344 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 04:25:29.543808 16078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: Bootstrap starting.
I20260807 04:25:29.544332 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:25:29.547196 16078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:29.548373 16078 log.cc:824] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:29.551422 16078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: No bootstrap required, opened a new log
I20260807 04:25:29.563071 16078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.563534 16078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:29.563670 16078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0784ce0675ae4f9281dacf24651d6346, State: Initialized, Role: FOLLOWER
I20260807 04:25:29.564410 16078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.571396 15920 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 4
I20260807 04:25:29.571696 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.549041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.1.244.254:36495)
0807 04:25:29.549253 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.549260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.549560 (+ 300us) server_negotiation.cc:407] Connection header received
0807 04:25:29.549787 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.549792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.549890 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.550001 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.552883 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.553966 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.556622 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.557521 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.557648 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.563529 (+ 5881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.563535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.563540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.563583 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.568265 (+ 4682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.569181 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.569188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.569193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.569354 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.569831 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.569836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.569840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.570195 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.570444 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.571049 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.571314 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:25:29.576490 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.577997 16078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260807 04:25:29.585144 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.556087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.1.244.252:45959)
0807 04:25:29.556299 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.556305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.563285 (+ 6980us) server_negotiation.cc:407] Connection header received
0807 04:25:29.563815 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.563822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.563923 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.564012 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.566183 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.567359 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.569505 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.570221 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.570332 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.577982 (+ 7650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.577990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.577998 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.578034 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.581478 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.582093 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.582098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.582102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.582287 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.582848 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.582854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.582858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.583238 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.583492 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.584547 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.584788 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:25:29.585417 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.549838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.1.244.253:42845)
0807 04:25:29.549997 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.550004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.562893 (+ 12889us) server_negotiation.cc:407] Connection header received
0807 04:25:29.563423 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.563430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.563522 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.563622 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.566340 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.567793 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.570596 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.571494 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.571614 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.577438 (+ 5824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.577445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.577449 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.577494 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.582029 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.582926 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.582933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.582936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.583079 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.583534 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.583539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.583542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.584024 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.584241 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.584828 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.585050 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 04:25:29.586951 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:25:29.593456 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.571943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57872)
0807 04:25:29.572268 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.572287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:29.572379 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:29.573521 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:29.573527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:29.574061 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:29.574285 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.574300 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.576167 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.576175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.577754 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.577767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.578861 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.578869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.579073 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.582339 (+ 3266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:29.582385 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:29.586665 (+ 4280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:29.590251 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.590256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.590260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.591009 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.591632 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.591640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.591643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.591825 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.592468 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:29.592473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:29.592653 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 04:25:29.592796 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 04:25:29.593086 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260807 04:25:29.594599 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.572563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.1.244.252:45959)
0807 04:25:29.573195 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.573200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.573221 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:25:29.573265 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.573269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.573336 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.574464 (+ 1128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.574745 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.576002 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.577939 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.578687 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.579338 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.586950 (+ 7612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.586956 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.586960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.586994 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.590108 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.591265 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.591273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.591276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.591479 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.592016 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.592021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.592029 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.592325 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.592504 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.593812 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.594063 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":3037,"threads_started":1}
W20260807 04:25:29.599078 16107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:25:29.599205 16107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:25:29.602622 15920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: Bootstrap starting.
I20260807 04:25:29.606455 15920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:29.607725 15920 log.cc:824] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:29.593357 16007 master_runner.cc:186] Error getting master registration for 127.1.244.254:36495: OK, instance_id { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" instance_seqno: 1786076728435536 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:25:29.612600 16007 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:25:29.613474 15920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: No bootstrap required, opened a new log
I20260807 04:25:29.624963 15920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.625380 15920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:29.625514 15920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315f1cbdf6cd43c0a001317864341e73, State: Initialized, Role: FOLLOWER
I20260807 04:25:29.626195 15920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.628263 16110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.628724 16110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:25:29.629340 15920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:25:29.644835 16125 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:25:29.644974 16125 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:25:29.638372 15851 master_runner.cc:186] Error getting master registration for 127.1.244.253:42845: OK, instance_id { permanent_uuid: "5366a4e646f147909701d4badebc5473" instance_seqno: 1786076728929625 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:25:29.646967 15851 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:25:29.655123 16000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } attempt: 5
Aug 07 04:25:29 dist-test-slave-zkpd krb5kdc[15837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076728, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260807 04:25:29.672778 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.655672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45959 (local address 127.0.0.1:57886)
0807 04:25:29.655950 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:29.655971 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:29.656086 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:29.656477 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:29.656482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:29.657041 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:29.657255 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.657269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.658764 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.658769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.660218 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:29.660234 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.661238 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.661244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:29.661474 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.662196 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:29.662220 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:29.665794 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:29.669945 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.669951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.669955 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.670501 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.670923 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:29.670928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:29.670932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:29.671121 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:29.671735 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:29.671740 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:29.671942 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 04:25:29.672108 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 04:25:29.672394 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 04:25:29.672829 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:29.655751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.1.244.252:45959)
0807 04:25:29.655940 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:29.655948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:29.656055 (+ 107us) server_negotiation.cc:407] Connection header received
0807 04:25:29.656232 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:29.656237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:29.656329 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:29.656431 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:29.657423 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.658618 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.660397 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:29.661077 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:29.661210 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:29.665996 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:29.666003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:29.666007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:29.666043 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:29.669809 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.670635 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.670641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.670644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.670799 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:29.671244 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:29.671249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:29.671252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:29.671605 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:29.671760 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:29.672239 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 04:25:29.672489 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 04:25:29.674357 15872 tablet.cc:2369] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:29.680037 16000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473: Bootstrap starting.
I20260807 04:25:29.683930 16000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:29.685151 16000 log.cc:824] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:29.689416 16000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473: No bootstrap required, opened a new log
I20260807 04:25:29.701692 16000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.702189 16000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:29.702304 16000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5366a4e646f147909701d4badebc5473, State: Initialized, Role: FOLLOWER
I20260807 04:25:29.703006 16000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.704041 16128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.704514 16128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 04:25:29.705319 16030 tablet.cc:2369] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:29.707048 16000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:29.727720 15927 master_runner.cc:263] No leader master found from master 5366a4e646f147909701d4badebc5473
I20260807 04:25:29.727851 15927 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 04:25:29.730497 16141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:25:29.730641 16141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:25:29.794284 16128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:25:29.794603 16128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.796484 16128 leader_election.cc:288] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 315f1cbdf6cd43c0a001317864341e73 (127.1.244.254:36495), 0784ce0675ae4f9281dacf24651d6346 (127.1.244.252:45959)
I20260807 04:25:29.797821 15894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5366a4e646f147909701d4badebc5473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315f1cbdf6cd43c0a001317864341e73" is_pre_election: true
I20260807 04:25:29.798437 15894 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5366a4e646f147909701d4badebc5473 in term 0.
I20260807 04:25:29.799130 16052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5366a4e646f147909701d4badebc5473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0784ce0675ae4f9281dacf24651d6346" is_pre_election: true
I20260807 04:25:29.799711 16052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5366a4e646f147909701d4badebc5473 in term 0.
I20260807 04:25:29.800547 15943 leader_election.cc:302] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315f1cbdf6cd43c0a001317864341e73, 5366a4e646f147909701d4badebc5473; no voters:
I20260807 04:25:29.801834 16128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:25:29.802018 16128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:25:29.802142 16128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:29.809719 16128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.812877 15894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5366a4e646f147909701d4badebc5473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315f1cbdf6cd43c0a001317864341e73"
I20260807 04:25:29.813141 15894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:29.816795 15894 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5366a4e646f147909701d4badebc5473 in term 1.
I20260807 04:25:29.817263 15943 leader_election.cc:302] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315f1cbdf6cd43c0a001317864341e73, 5366a4e646f147909701d4badebc5473; no voters:
I20260807 04:25:29.820888 16052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5366a4e646f147909701d4badebc5473" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0784ce0675ae4f9281dacf24651d6346"
I20260807 04:25:29.821123 16052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:29.824620 16052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5366a4e646f147909701d4badebc5473 in term 1.
I20260807 04:25:29.825300 16128 leader_election.cc:288] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [CANDIDATE]: Term 1 election: Requested vote from peers 315f1cbdf6cd43c0a001317864341e73 (127.1.244.254:36495), 0784ce0675ae4f9281dacf24651d6346 (127.1.244.252:45959)
I20260807 04:25:29.825678 16128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:29.826072 16128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [term 1 LEADER]: Becoming Leader. State: Replica: 5366a4e646f147909701d4badebc5473, State: Running, Role: LEADER
I20260807 04:25:29.827299 16128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } }
I20260807 04:25:29.844640 16143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5366a4e646f147909701d4badebc5473. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.845139 16143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:29.855873 16146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:29.858227 16146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:29.877091 16052 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [term 1 FOLLOWER]: Refusing update from remote peer 5366a4e646f147909701d4badebc5473: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:29.878100 16143 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:29.885710 15894 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [term 1 FOLLOWER]: Refusing update from remote peer 5366a4e646f147909701d4badebc5473: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:29.886461 16143 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [LEADER]: Connected to new peer: Peer: permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:29.900545 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5366a4e646f147909701d4badebc5473. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.900888 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:25:29.907824 16110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5366a4e646f147909701d4badebc5473. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.908159 16110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:25:29.909991 16128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.910233 16146 catalog_manager.cc:1347] Generated new cluster ID: 75eeb9a952b9462b84b5ecb42393d013
I20260807 04:25:29.910324 16128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:29.910496 16146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:25:29.920748 16095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.921015 16095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:25:29.921268 16143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.921584 16143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473 [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:29.928961 16147 mvcc.cc:205] Tried to move back new op lower bound from 7315770285564653568 to 7315770285397602304. Current Snapshot: MvccSnapshot[applied={T|T < 7315770285564653568 or (T in {7315770285564653568})}]
I20260807 04:25:29.939471 16110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5366a4e646f147909701d4badebc5473" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "315f1cbdf6cd43c0a001317864341e73" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36495 } } peers { permanent_uuid: "5366a4e646f147909701d4badebc5473" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 42845 } } peers { permanent_uuid: "0784ce0675ae4f9281dacf24651d6346" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45959 } } }
I20260807 04:25:29.939733 16110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:25:29.960768 16146 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:25:29.962558 16146 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:25:29.995910 16146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5366a4e646f147909701d4badebc5473: Generated new TSK 0
I20260807 04:25:29.997102 16146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:25:30.601352 16107 catalog_manager.cc:1259] Loaded cluster ID: 75eeb9a952b9462b84b5ecb42393d013
I20260807 04:25:30.601531 16107 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: loading cluster ID for follower catalog manager: success
I20260807 04:25:30.604151 16107 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: acquiring CA information for follower catalog manager: success
I20260807 04:25:30.605903 16107 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0784ce0675ae4f9281dacf24651d6346: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:25:30.647140 16125 catalog_manager.cc:1259] Loaded cluster ID: 75eeb9a952b9462b84b5ecb42393d013
I20260807 04:25:30.647243 16125 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: loading cluster ID for follower catalog manager: success
I20260807 04:25:30.649603 16125 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: acquiring CA information for follower catalog manager: success
I20260807 04:25:30.651278 16125 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 315f1cbdf6cd43c0a001317864341e73: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:25:31.141083 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15851
I20260807 04:25:31.142308 15915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:31.250483 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15851
I20260807 04:25:31.274554 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 15927
I20260807 04:25:31.275648 15994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:31.385435 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 15927
I20260807 04:25:31.410293 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16007
I20260807 04:25:31.411415 16073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:31.523059 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16007
2026-08-07T04:25:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4925 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:25:31 dist-test-slave-zkpd krb5kdc[16169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:25:31 dist-test-slave-zkpd krb5kdc[16169](info): set up 2 sockets
Aug 07 04:25:31 dist-test-slave-zkpd krb5kdc[16169](info): commencing operation
krb5kdc: starting...
W20260807 04:25:32.643282 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:25:32 dist-test-slave-zkpd krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:25:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:25:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:25:32.891566 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35535
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:33.218631 16183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:33.218902 16183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:33.219069 16183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:33.226115 16183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:33.226208 16183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:33.226253 16183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:33.226328 16183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:33.226390 16183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:33.235795 16183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:33.237068 16183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:33.238344 16183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:33.250006 16188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:33.250046 16191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:33.249961 16189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:33.251049 16183 server_base.cc:1034] running on GCE node
I20260807 04:25:33.251845 16183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:33.253232 16183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:33.254470 16183 hybrid_clock.cc:648] HybridClock initialized: now 1786076733254436 us; error 43 us; skew 500 ppm
I20260807 04:25:33.254810 16183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:33 dist-test-slave-zkpd krb5kdc[16169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076733, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:33.258008 16183 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:33.259456 16183 webserver.cc:460] Webserver started at http://127.1.244.254:41195/ using document root <none> and password file <none>
I20260807 04:25:33.260190 16183 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:33.260340 16183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:33.260658 16183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:33.263523 16183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "07e3a19f61524dce8831ca749a32b20a"
format_stamp: "Formatted at 2026-08-07 04:25:33 on dist-test-slave-zkpd"
server_key: "d57ae0b6efc3fd9e8c9666ef2d03c2ba"
server_key_iv: "cb6739472421a87f3f2ad126dc3ae6de"
server_key_version: "encryptionkey@0"
I20260807 04:25:33.264289 16183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "07e3a19f61524dce8831ca749a32b20a"
format_stamp: "Formatted at 2026-08-07 04:25:33 on dist-test-slave-zkpd"
server_key: "d57ae0b6efc3fd9e8c9666ef2d03c2ba"
server_key_iv: "cb6739472421a87f3f2ad126dc3ae6de"
server_key_version: "encryptionkey@0"
I20260807 04:25:33.269130 16183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:25:33.272487 16198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:33.273777 16183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:25:33.273958 16183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "07e3a19f61524dce8831ca749a32b20a"
format_stamp: "Formatted at 2026-08-07 04:25:33 on dist-test-slave-zkpd"
server_key: "d57ae0b6efc3fd9e8c9666ef2d03c2ba"
server_key_iv: "cb6739472421a87f3f2ad126dc3ae6de"
server_key_version: "encryptionkey@0"
I20260807 04:25:33.274155 16183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:33.297992 16183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:33.301846 16183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:33.302134 16183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:33.320772 16183 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35535
I20260807 04:25:33.320767 16252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35535 every 8 connection(s)
I20260807 04:25:33.322099 16183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:33.323767 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16183
I20260807 04:25:33.324029 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:25:33.324752 2003 external_mini_cluster.cc:1428] Setting key ff50ca9cc5e9d7b4a6bc4cc50729e890
I20260807 04:25:33.326181 16253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:25:33 dist-test-slave-zkpd krb5kdc[16169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:33.339687 16253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a: Bootstrap starting.
I20260807 04:25:33.343820 16253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:33.345212 16253 log.cc:824] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:33.348718 16253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a: No bootstrap required, opened a new log
I20260807 04:25:33.349619 16257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:33.326596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.1.244.254:35535)
0807 04:25:33.327464 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:33.327477 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:33.327525 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:25:33.329524 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:33.329551 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:33.330050 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:33.330483 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:33.331871 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:33.333960 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:33.335255 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:33.336271 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:33.336463 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:33.340741 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:33.340751 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:33.340769 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:33.340815 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:33.344820 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:33.345670 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:33.345681 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:33.345690 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:33.345954 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:33.346418 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:33.346421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:33.346423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:33.347099 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:33.347366 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:33.347752 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 04:25:33.348234 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":363,"threads_started":1}
I20260807 04:25:33.353343 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:25:33.356891 16253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e3a19f61524dce8831ca749a32b20a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35535 } }
I20260807 04:25:33.357236 16253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:33.357353 16253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e3a19f61524dce8831ca749a32b20a, State: Initialized, Role: FOLLOWER
I20260807 04:25:33.357883 16253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e3a19f61524dce8831ca749a32b20a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35535 } }
I20260807 04:25:33.358106 16253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:25:33.358266 16253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:25:33.358428 16253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:33.360586 16253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e3a19f61524dce8831ca749a32b20a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35535 } }
I20260807 04:25:33.361049 16253 leader_election.cc:302] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07e3a19f61524dce8831ca749a32b20a; no voters:
I20260807 04:25:33.361630 16253 leader_election.cc:288] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:25:33.361769 16259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:33.362548 16259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [term 1 LEADER]: Becoming Leader. State: Replica: 07e3a19f61524dce8831ca749a32b20a, State: Running, Role: LEADER
I20260807 04:25:33.363062 16259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e3a19f61524dce8831ca749a32b20a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35535 } }
I20260807 04:25:33.363691 16253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:33.366031 16261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e3a19f61524dce8831ca749a32b20a. Latest consensus state: current_term: 1 leader_uuid: "07e3a19f61524dce8831ca749a32b20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e3a19f61524dce8831ca749a32b20a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35535 } } }
I20260807 04:25:33.366513 16261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:33.366937 16260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07e3a19f61524dce8831ca749a32b20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e3a19f61524dce8831ca749a32b20a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35535 } } }
I20260807 04:25:33.372115 16260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:33.373719 16271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:33.378729 16271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:33.386598 16271 catalog_manager.cc:1347] Generated new cluster ID: f9857bce903b46b4a688404ac8351791
I20260807 04:25:33.386703 16271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:25:33.394105 16271 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:25:33.395833 16271 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:25:33.413900 16271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07e3a19f61524dce8831ca749a32b20a: Generated new TSK 0
I20260807 04:25:33.414835 16271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:25:33.466467 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16183
I20260807 04:25:33.467592 16248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:33.561146 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16183
2026-08-07T04:25:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2034 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:25:33 dist-test-slave-zkpd krb5kdc[16287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:25:33 dist-test-slave-zkpd krb5kdc[16287](info): set up 2 sockets
Aug 07 04:25:33 dist-test-slave-zkpd krb5kdc[16287](info): commencing operation
krb5kdc: starting...
W20260807 04:25:34.723132 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:25:34 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:25:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:25:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:25:34.972827 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41841
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:35.308123 16300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:35.308462 16300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:35.308617 16300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:35.316466 16300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:35.316604 16300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:35.316704 16300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:35.316807 16300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:35.316885 16300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:35.326794 16300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:35.328214 16300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:35.329576 16300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:35.342157 16305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:35.342092 16308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:35.342135 16306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:35.343096 16300 server_base.cc:1034] running on GCE node
I20260807 04:25:35.343941 16300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:35.345367 16300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:35.346589 16300 hybrid_clock.cc:648] HybridClock initialized: now 1786076735346532 us; error 51 us; skew 500 ppm
I20260807 04:25:35.346932 16300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:35 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076735, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:35.350119 16300 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:35.351634 16300 webserver.cc:460] Webserver started at http://127.1.244.254:43239/ using document root <none> and password file <none>
I20260807 04:25:35.352358 16300 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:35.352490 16300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:35.352806 16300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:35.355815 16300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "6a1776690e684ae9967c78c8ba3f5aad"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "9cecfa151235a6c57e501b93d773dc34"
server_key_iv: "edeb8e40bebb58a7c432ac80dbc6e51a"
server_key_version: "encryptionkey@0"
I20260807 04:25:35.356758 16300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "6a1776690e684ae9967c78c8ba3f5aad"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "9cecfa151235a6c57e501b93d773dc34"
server_key_iv: "edeb8e40bebb58a7c432ac80dbc6e51a"
server_key_version: "encryptionkey@0"
I20260807 04:25:35.362059 16300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:25:35.365660 16315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:35.367093 16300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:25:35.367274 16300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "6a1776690e684ae9967c78c8ba3f5aad"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "9cecfa151235a6c57e501b93d773dc34"
server_key_iv: "edeb8e40bebb58a7c432ac80dbc6e51a"
server_key_version: "encryptionkey@0"
I20260807 04:25:35.367473 16300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:35.381690 16300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:35.385457 16300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:35.385800 16300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:35.404510 16300 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41841
I20260807 04:25:35.404541 16369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41841 every 8 connection(s)
I20260807 04:25:35.405913 16300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:35.406383 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16300
I20260807 04:25:35.406620 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:25:35.407564 2003 external_mini_cluster.cc:1428] Setting key b6c6d03f381f8cef547a31b9fd59f61e
I20260807 04:25:35.410495 16370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:25:35 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:35.423276 16370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap starting.
I20260807 04:25:35.426498 16370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:35.427619 16370 log.cc:824] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:35.430763 16370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: No bootstrap required, opened a new log
I20260807 04:25:35.431521 16373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:35.409690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.1.244.254:41841)
0807 04:25:35.410560 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:35.410576 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:35.410626 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:25:35.412939 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:35.412965 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:35.413439 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:35.413920 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:35.415195 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:35.417293 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:35.418640 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:35.419652 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:35.419852 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:35.424263 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:35.424270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:35.424282 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:35.424315 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:35.427361 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:35.428208 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:35.428214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:35.428219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:35.428374 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:35.428819 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:35.428825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:35.428834 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:35.429311 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:35.429667 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:35.430065 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 04:25:35.430486 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":366,"threads_started":1}
I20260807 04:25:35.438213 16370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:35.438619 16370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:35.438710 16370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a1776690e684ae9967c78c8ba3f5aad, State: Initialized, Role: FOLLOWER
I20260807 04:25:35.439231 16370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:35.439469 16370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:25:35.439636 16370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:25:35.439806 16370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:35.442126 16370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:35.442595 16370 leader_election.cc:302] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a1776690e684ae9967c78c8ba3f5aad; no voters:
I20260807 04:25:35.443156 16370 leader_election.cc:288] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:25:35.443327 16376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:35.444101 16376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 LEADER]: Becoming Leader. State: Replica: 6a1776690e684ae9967c78c8ba3f5aad, State: Running, Role: LEADER
I20260807 04:25:35.444626 16376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:35.445324 16370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:35.447594 16377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a1776690e684ae9967c78c8ba3f5aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } } }
I20260807 04:25:35.447568 16378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a1776690e684ae9967c78c8ba3f5aad. Latest consensus state: current_term: 1 leader_uuid: "6a1776690e684ae9967c78c8ba3f5aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } } }
I20260807 04:25:35.448017 16377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:35.448062 16378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:35.457231 16382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:35.463229 16382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:35.474068 16382 catalog_manager.cc:1347] Generated new cluster ID: 8afed9de62584979b8115fd0c33ecf31
I20260807 04:25:35.474169 16382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:25:35.486970 16382 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:25:35.488812 16382 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:25:35.503661 16382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Generated new TSK 0
I20260807 04:25:35.504700 16382 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:25:35.576979 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41841
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:35.909866 16397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:35.910164 16397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:35.910336 16397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:35.917343 16397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:35.917472 16397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:35.917642 16397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:25:35.926932 16397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:35.928102 16397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:35.929379 16397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:35.942354 16406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:35.942348 16403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:35.942364 16404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:35.943150 16397 server_base.cc:1034] running on GCE node
I20260807 04:25:35.943708 16397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:35.944618 16397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:35.945863 16397 hybrid_clock.cc:648] HybridClock initialized: now 1786076735945805 us; error 60 us; skew 500 ppm
I20260807 04:25:35.946197 16397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:35 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076735, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:35.949329 16397 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:25:35.950871 16397 webserver.cc:460] Webserver started at http://127.1.244.193:35061/ using document root <none> and password file <none>
I20260807 04:25:35.951550 16397 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:35.951692 16397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:35.952054 16397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:35.954813 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "f786f020d78446e9b0ac9729e25a7879"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "13e96597473fa95c8341637f880e0bcf"
server_key_iv: "aa47107da6252ef1a2b21c87afc1d4ca"
server_key_version: "encryptionkey@0"
I20260807 04:25:35.955557 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "f786f020d78446e9b0ac9729e25a7879"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "13e96597473fa95c8341637f880e0bcf"
server_key_iv: "aa47107da6252ef1a2b21c87afc1d4ca"
server_key_version: "encryptionkey@0"
I20260807 04:25:35.960475 16397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:25:35.963742 16413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:35.964993 16397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:25:35.965161 16397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "f786f020d78446e9b0ac9729e25a7879"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "13e96597473fa95c8341637f880e0bcf"
server_key_iv: "aa47107da6252ef1a2b21c87afc1d4ca"
server_key_version: "encryptionkey@0"
I20260807 04:25:35.965360 16397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:35.985208 16397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:35.988876 16397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:35.989151 16397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:35.990012 16397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:35.991245 16397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:35.991321 16397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:35.991479 16397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:35.991542 16397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:36.023658 16397 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44215
I20260807 04:25:36.023698 16534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44215 every 8 connection(s)
I20260807 04:25:36.025071 16397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:25:36.033957 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16397
I20260807 04:25:36.034183 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:25:36.034878 2003 external_mini_cluster.cc:1428] Setting key 39c34fbd6d158376a96b4955a22421e5
Aug 07 04:25:36 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076735, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:36.047060 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:36.027617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:34601 (local address 127.1.244.254:41841)
0807 04:25:36.028121 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:36.028126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:36.029077 (+ 951us) server_negotiation.cc:407] Connection header received
0807 04:25:36.031116 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:36.031120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:36.031188 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:36.031283 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:36.032763 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.033483 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.035180 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.035699 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.035841 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:36.039758 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:36.039767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:36.039775 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:36.039820 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:36.043620 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:36.044461 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:36.044469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:36.044476 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:36.044631 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:36.045074 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:36.045082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:36.045087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:36.045448 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:36.045601 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:36.046277 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 04:25:36.046525 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":263,"threads_started":1}
I20260807 04:25:36.047744 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:36.028152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:34601)
0807 04:25:36.028898 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:36.028948 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:36.030866 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:36.031496 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:36.031503 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:36.032058 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:36.032617 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:36.032635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.033638 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.033641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:36.035032 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:36.035045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.035959 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.035971 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:36.036215 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:36.036925 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:36.036963 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:36.039550 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:36.043757 (+ 4207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:36.043764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:36.043776 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:36.044295 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:36.044749 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:36.044753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:36.044755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:36.044956 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:36.045583 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:36.045588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:36.045886 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 04:25:36.046104 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 04:25:36.046576 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":271,"threads_started":1}
I20260807 04:25:36.049433 16535 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41841
I20260807 04:25:36.049755 16535 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:36.050665 16535 heartbeater.cc:511] Master 127.1.244.254:41841 requested a full tablet report, sending...
I20260807 04:25:36.052876 16334 ts_manager.cc:194] Registered new tserver with Master: f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215)
I20260807 04:25:36.054351 16334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:34601
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:25:36.095665 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41841
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:36.424556 16542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:36.424850 16542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:36.425015 16542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:36.432507 16542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:36.432643 16542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:36.432817 16542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:25:36.442126 16542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:36.443356 16542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:36.444664 16542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:36.457683 16548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:36.457773 16550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:36.457679 16547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:36.458527 16542 server_base.cc:1034] running on GCE node
I20260807 04:25:36.459102 16542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:36.460014 16542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:36.461241 16542 hybrid_clock.cc:648] HybridClock initialized: now 1786076736461197 us; error 52 us; skew 500 ppm
I20260807 04:25:36.461570 16542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:36 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:36.464742 16542 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:25:36.466174 16542 webserver.cc:460] Webserver started at http://127.1.244.194:45949/ using document root <none> and password file <none>
I20260807 04:25:36.466868 16542 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:36.467010 16542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:36.467315 16542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:36.470372 16542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "17f66e1481ad49dbab502ccadb055c21"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "fbf6a269dcd02e052d74a574a6917eeb"
server_key_iv: "893c96314ecc64d7bcb7ee382b7635f3"
server_key_version: "encryptionkey@0"
I20260807 04:25:36.471204 16542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "17f66e1481ad49dbab502ccadb055c21"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "fbf6a269dcd02e052d74a574a6917eeb"
server_key_iv: "893c96314ecc64d7bcb7ee382b7635f3"
server_key_version: "encryptionkey@0"
I20260807 04:25:36.476138 16542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:25:36.479362 16558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:36.480620 16542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:25:36.480805 16542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "17f66e1481ad49dbab502ccadb055c21"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "fbf6a269dcd02e052d74a574a6917eeb"
server_key_iv: "893c96314ecc64d7bcb7ee382b7635f3"
server_key_version: "encryptionkey@0"
I20260807 04:25:36.481016 16542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:36.493582 16542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:36.497666 16542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:36.497951 16542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:36.498930 16542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:36.500289 16542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:36.500387 16542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:36.500530 16542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:36.500602 16542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:36.533465 16542 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45085
I20260807 04:25:36.533545 16674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45085 every 8 connection(s)
I20260807 04:25:36.534741 16542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:25:36.539595 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16542
I20260807 04:25:36.539784 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:25:36.540254 2003 external_mini_cluster.cc:1428] Setting key d1dc8843f6fa042f075e8f5e8cbb54c1
Aug 07 04:25:36 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:36.556090 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:36.537205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:36335 (local address 127.1.244.254:41841)
0807 04:25:36.537426 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:36.537434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:36.538580 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 04:25:36.540575 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:36.540581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:36.540670 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:36.540791 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:36.542850 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.543543 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.545158 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.545596 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.545729 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:36.549224 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:36.549232 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:36.549240 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:36.549282 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:36.552869 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:36.553822 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:36.553826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:36.553828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:36.553958 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:36.554435 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:36.554438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:36.554440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:36.554688 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:36.554794 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:36.555610 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 04:25:36.555764 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:25:36.557541 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:36.537628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:36335)
0807 04:25:36.538406 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:36.538449 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:36.540281 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:36.541272 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:36.541283 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:36.541971 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:36.542677 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:36.542695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.543694 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.543700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:36.544996 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:36.545008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:36.545837 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:36.545845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:36.546100 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:36.546844 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:36.546865 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:36.549010 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:36.553036 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:36.553045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:36.553061 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:36.553670 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:36.554066 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:36.554074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:36.554077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:36.554321 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:36.554817 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:36.554828 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:36.555248 (+ 420us) client_negotiation.cc:769] Sending connection context
0807 04:25:36.555522 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 04:25:36.556105 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":344,"threads_started":1}
I20260807 04:25:36.558955 16675 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41841
I20260807 04:25:36.559250 16675 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:36.559998 16675 heartbeater.cc:511] Master 127.1.244.254:41841 requested a full tablet report, sending...
I20260807 04:25:36.561434 16334 ts_manager.cc:194] Registered new tserver with Master: 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194:45085)
I20260807 04:25:36.562216 16334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:36335
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:25:36.599771 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41841
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:36.924870 16680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:36.925179 16680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:36.925351 16680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:36.932505 16680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:36.932646 16680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:36.932837 16680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:25:36.942205 16680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:36.943451 16680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:36.944787 16680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:36.958227 16688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:36.958329 16686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:36.958297 16685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:36.959168 16680 server_base.cc:1034] running on GCE node
I20260807 04:25:36.959748 16680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:36.960813 16680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:36.962013 16680 hybrid_clock.cc:648] HybridClock initialized: now 1786076736961950 us; error 51 us; skew 500 ppm
I20260807 04:25:36.962268 16680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:36 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:36.965387 16680 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:25:36.966796 16680 webserver.cc:460] Webserver started at http://127.1.244.195:42679/ using document root <none> and password file <none>
I20260807 04:25:36.967401 16680 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:36.967509 16680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:36.967746 16680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:25:36.970646 16680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "c3e9a236166f432696c72afb87ff0468"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "58da0a0202cf3a068cc4c68af86ead90"
server_key_iv: "6a9bda6cc97f706bdb4a65fe2e4445b9"
server_key_version: "encryptionkey@0"
I20260807 04:25:36.971415 16680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "c3e9a236166f432696c72afb87ff0468"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "58da0a0202cf3a068cc4c68af86ead90"
server_key_iv: "6a9bda6cc97f706bdb4a65fe2e4445b9"
server_key_version: "encryptionkey@0"
I20260807 04:25:36.976433 16680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 04:25:36.979727 16695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:36.981024 16680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:25:36.981196 16680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "c3e9a236166f432696c72afb87ff0468"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "58da0a0202cf3a068cc4c68af86ead90"
server_key_iv: "6a9bda6cc97f706bdb4a65fe2e4445b9"
server_key_version: "encryptionkey@0"
I20260807 04:25:36.981360 16680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:37.008507 16680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:37.012853 16680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:37.013146 16680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:37.014101 16680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:37.015440 16680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:37.015513 16680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:37.015689 16680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:37.015743 16680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:37.049502 16680 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:33183
I20260807 04:25:37.049535 16816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:33183 every 8 connection(s)
I20260807 04:25:37.050836 16680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:25:37.053471 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16680
I20260807 04:25:37.053679 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:25:37.054245 2003 external_mini_cluster.cc:1428] Setting key 72f0202828e5102ca6eeeca0d24487ba
I20260807 04:25:37.057942 16535 heartbeater.cc:503] Master 127.1.244.254:41841 was elected leader, sending a full tablet report...
Aug 07 04:25:37 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076736, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:37.071336 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.054795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56753 (local address 127.1.244.254:41841)
0807 04:25:37.054989 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.054995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.055206 (+ 211us) server_negotiation.cc:407] Connection header received
0807 04:25:37.057371 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.057376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.057477 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.057587 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:37.059230 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.060062 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.061649 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.062058 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.062192 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.065478 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:37.065483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:37.065486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:37.065516 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:37.067968 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.068801 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.068807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.068814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.068974 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.069535 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.069540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.069544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.069979 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:37.070152 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.070830 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.071066 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:25:37.072382 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.054177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:56753)
0807 04:25:37.055067 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:37.055103 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:37.057070 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:37.057827 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:37.057835 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:37.058387 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:37.059064 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.059083 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.060227 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.060233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.061497 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.061509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.062289 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.062297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.062530 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.063131 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:37.063152 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:37.065274 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:37.068101 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:37.068109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:37.068123 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:37.068619 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:37.069199 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:37.069204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:37.069207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:37.069440 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:37.070134 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:37.070140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:37.070518 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 04:25:37.070747 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 04:25:37.071218 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":346,"threads_started":1}
I20260807 04:25:37.073613 16817 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41841
I20260807 04:25:37.073931 16817 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:37.074601 16817 heartbeater.cc:511] Master 127.1.244.254:41841 requested a full tablet report, sending...
I20260807 04:25:37.075873 16334 ts_manager.cc:194] Registered new tserver with Master: c3e9a236166f432696c72afb87ff0468 (127.1.244.195:33183)
I20260807 04:25:37.076679 16334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:56753
I20260807 04:25:37.082017 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:25:37.099428 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.086682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.1.244.254:41841)
0807 04:25:37.086865 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.086873 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.087245 (+ 372us) server_negotiation.cc:407] Connection header received
0807 04:25:37.087449 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.087454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.087521 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.087638 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:37.089107 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.089825 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.091218 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.091688 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.091825 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.094281 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:37.094292 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:37.094295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:37.094326 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:37.096685 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.097368 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.097374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.097377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.097517 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.097913 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.097920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.097924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.098245 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:37.098512 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.098874 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.099093 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 04:25:37.103408 16334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:25:37.110193 16334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:25:37.135699 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.122033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:33183 (local address 127.0.0.1:42284)
0807 04:25:37.122850 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:37.122895 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:37.123062 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:37.124840 (+ 1778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:37.124849 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:37.124886 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:37.125332 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.125347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.128499 (+ 3152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.128507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.131733 (+ 3226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.131752 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.134251 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.134261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.134579 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.134656 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 04:25:37.134787 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 04:25:37.135042 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":27,"thread_start_us":2318,"threads_started":1}
I20260807 04:25:37.137157 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.122268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.1.244.195:33183)
0807 04:25:37.124275 (+ 2007us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.124280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.124305 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:25:37.124382 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.124387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.124561 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.125590 (+ 1029us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:37.125978 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.128285 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.131963 (+ 3678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.133972 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.136381 (+ 2409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.136642 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.136724 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.136841 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":1741,"threads_started":1}
I20260807 04:25:37.137552 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.123316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33662 (local address 127.1.244.194:45085)
0807 04:25:37.123978 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.123987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.124910 (+ 923us) server_negotiation.cc:407] Connection header received
0807 04:25:37.125046 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.125056 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.125319 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.125552 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:37.126745 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.128850 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.134200 (+ 5350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.136119 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.136286 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.136531 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.137062 (+ 531us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.137247 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":326,"threads_started":1}
I20260807 04:25:37.137665 16829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.122116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:45085 (local address 127.0.0.1:33662)
0807 04:25:37.124640 (+ 2524us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:37.124660 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:37.124772 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:37.125851 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:37.125858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:37.125901 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:37.126450 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.126465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.129061 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.129069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.133971 (+ 4902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.133990 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.136673 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.136679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.136854 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.136885 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:25:37.136985 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:25:37.137106 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2374,"mutex_wait_us":1,"thread_start_us":452,"threads_started":1}
I20260807 04:25:37.142204 16608 tablet_service.cc:1467] Processing CreateTablet for tablet 0788826e596840269cb1418c3c6958d3 (DEFAULT_TABLE table=test-table [id=9bd39b471b0946ffbaca8a5169f0aa47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:37.142369 16747 tablet_service.cc:1467] Processing CreateTablet for tablet 0788826e596840269cb1418c3c6958d3 (DEFAULT_TABLE table=test-table [id=9bd39b471b0946ffbaca8a5169f0aa47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:37.143549 16608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0788826e596840269cb1418c3c6958d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:37.143790 16747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0788826e596840269cb1418c3c6958d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:37.145371 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.124749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44215 (local address 127.0.0.1:49768)
0807 04:25:37.136028 (+ 11279us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:37.136052 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:37.136176 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:37.138034 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:37.138038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:37.138067 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:37.138237 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.138247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.140424 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.140429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.142767 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.142782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.144624 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.144633 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.144822 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.144859 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:25:37.144940 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 04:25:37.145070 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11122,"thread_start_us":12888,"threads_started":1}
I20260807 04:25:37.146430 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.125195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.1.244.193:44215)
0807 04:25:37.136779 (+ 11584us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.136790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.136815 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:25:37.136933 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.136943 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.137167 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.138037 (+ 870us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:37.138396 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.140255 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.142970 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.144438 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.145620 (+ 1182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.145860 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.145952 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.146111 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11411,"thread_start_us":11247,"threads_started":1}
I20260807 04:25:37.151507 16466 tablet_service.cc:1467] Processing CreateTablet for tablet 0788826e596840269cb1418c3c6958d3 (DEFAULT_TABLE table=test-table [id=9bd39b471b0946ffbaca8a5169f0aa47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:37.152926 16466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0788826e596840269cb1418c3c6958d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:37.157218 16835 tablet_bootstrap.cc:492] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Bootstrap starting.
I20260807 04:25:37.160856 16835 tablet_bootstrap.cc:654] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:37.161303 16836 tablet_bootstrap.cc:492] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Bootstrap starting.
I20260807 04:25:37.162300 16835 log.cc:824] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:37.164431 16836 tablet_bootstrap.cc:654] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:37.165350 16836 log.cc:824] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:37.165475 16835 tablet_bootstrap.cc:492] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: No bootstrap required, opened a new log
I20260807 04:25:37.165746 16835 ts_tablet_manager.cc:1397] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 04:25:37.168979 16839 tablet_bootstrap.cc:492] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Bootstrap starting.
I20260807 04:25:37.170672 16836 tablet_bootstrap.cc:492] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: No bootstrap required, opened a new log
I20260807 04:25:37.170982 16836 ts_tablet_manager.cc:1397] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260807 04:25:37.172653 16839 tablet_bootstrap.cc:654] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:37.173893 16839 log.cc:824] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:37.175375 16835 raft_consensus.cc:348] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.175922 16835 raft_consensus.cc:374] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:37.176088 16835 raft_consensus.cc:729] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e9a236166f432696c72afb87ff0468, State: Initialized, Role: FOLLOWER
I20260807 04:25:37.177042 16839 tablet_bootstrap.cc:492] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: No bootstrap required, opened a new log
I20260807 04:25:37.176929 16835 consensus_queue.cc:260] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.177286 16839 ts_tablet_manager.cc:1397] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:25:37.178063 16817 heartbeater.cc:503] Master 127.1.244.254:41841 was elected leader, sending a full tablet report...
I20260807 04:25:37.178741 16835 ts_tablet_manager.cc:1428] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260807 04:25:37.181195 16836 raft_consensus.cc:348] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.181613 16836 raft_consensus.cc:374] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:37.181715 16836 raft_consensus.cc:729] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f66e1481ad49dbab502ccadb055c21, State: Initialized, Role: FOLLOWER
I20260807 04:25:37.182317 16836 consensus_queue.cc:260] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.183466 16675 heartbeater.cc:503] Master 127.1.244.254:41841 was elected leader, sending a full tablet report...
I20260807 04:25:37.184116 16836 ts_tablet_manager.cc:1428] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260807 04:25:37.187318 16839 raft_consensus.cc:348] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.187758 16839 raft_consensus.cc:374] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:37.187875 16839 raft_consensus.cc:729] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f786f020d78446e9b0ac9729e25a7879, State: Initialized, Role: FOLLOWER
I20260807 04:25:37.188472 16839 consensus_queue.cc:260] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.189913 16839 ts_tablet_manager.cc:1428] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260807 04:25:37.248503 16841 raft_consensus.cc:482] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:25:37.248833 16841 raft_consensus.cc:504] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.250689 16841 leader_election.cc:288] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194:45085), f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215)
I20260807 04:25:37.257231 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.251215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:36709 (local address 127.1.244.194:45085)
0807 04:25:37.251436 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.251443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.251464 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:25:37.251548 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.251555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.251633 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.251741 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:37.252205 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.253227 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.255212 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.256451 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.256580 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.256633 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.256956 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.257049 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:25:37.257344 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.251021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:45085 (local address 127.1.244.195:36709)
0807 04:25:37.251240 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:37.251261 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:37.251387 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:37.251752 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:37.251755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:37.251776 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:37.252030 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.252044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.253362 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.253367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.255049 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.255062 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.256596 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.256602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.256798 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.256833 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 04:25:37.256938 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 04:25:37.257069 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260807 04:25:37.258416 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.251631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44215 (local address 127.1.244.195:43373)
0807 04:25:37.252377 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:37.252391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:37.252490 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:37.252771 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:37.252774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:37.252790 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:37.253073 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.253079 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.254482 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.254486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.256318 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:37.256335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.257906 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.257913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:37.258031 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.258053 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 04:25:37.258124 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 04:25:37.258199 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":417,"threads_started":1}
I20260807 04:25:37.258582 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.251140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:43373 (local address 127.1.244.193:44215)
0807 04:25:37.251364 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.251369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.252484 (+ 1115us) server_negotiation.cc:407] Connection header received
0807 04:25:37.252586 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.252590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.252664 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.252756 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:37.253231 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.254331 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.256474 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.257708 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.257825 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.257895 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.258179 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.258311 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:25:37.258432 16629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0788826e596840269cb1418c3c6958d3" candidate_uuid: "c3e9a236166f432696c72afb87ff0468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f66e1481ad49dbab502ccadb055c21" is_pre_election: true
I20260807 04:25:37.259002 16629 raft_consensus.cc:2454] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e9a236166f432696c72afb87ff0468 in term 0.
I20260807 04:25:37.259582 16488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0788826e596840269cb1418c3c6958d3" candidate_uuid: "c3e9a236166f432696c72afb87ff0468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f786f020d78446e9b0ac9729e25a7879" is_pre_election: true
I20260807 04:25:37.259770 16699 leader_election.cc:302] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f66e1481ad49dbab502ccadb055c21, c3e9a236166f432696c72afb87ff0468; no voters:
I20260807 04:25:37.260160 16841 raft_consensus.cc:2781] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:25:37.260169 16488 raft_consensus.cc:2454] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3e9a236166f432696c72afb87ff0468 in term 0.
I20260807 04:25:37.260308 16841 raft_consensus.cc:482] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:25:37.260403 16841 raft_consensus.cc:3037] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:37.262646 16841 raft_consensus.cc:504] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.263309 16841 leader_election.cc:288] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [CANDIDATE]: Term 1 election: Requested vote from peers 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194:45085), f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215)
I20260807 04:25:37.263659 16629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0788826e596840269cb1418c3c6958d3" candidate_uuid: "c3e9a236166f432696c72afb87ff0468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17f66e1481ad49dbab502ccadb055c21"
I20260807 04:25:37.263777 16488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0788826e596840269cb1418c3c6958d3" candidate_uuid: "c3e9a236166f432696c72afb87ff0468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f786f020d78446e9b0ac9729e25a7879"
I20260807 04:25:37.263849 16629 raft_consensus.cc:3037] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:37.264008 16488 raft_consensus.cc:3037] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:37.266119 16629 raft_consensus.cc:2454] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e9a236166f432696c72afb87ff0468 in term 1.
I20260807 04:25:37.266171 16488 raft_consensus.cc:2454] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3e9a236166f432696c72afb87ff0468 in term 1.
I20260807 04:25:37.266513 16699 leader_election.cc:302] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f66e1481ad49dbab502ccadb055c21, c3e9a236166f432696c72afb87ff0468; no voters:
I20260807 04:25:37.266800 16841 raft_consensus.cc:2781] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:37.267302 16841 raft_consensus.cc:686] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 1 LEADER]: Becoming Leader. State: Replica: c3e9a236166f432696c72afb87ff0468, State: Running, Role: LEADER
I20260807 04:25:37.267732 16841 consensus_queue.cc:237] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:37.272248 16333 catalog_manager.cc:5495] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3e9a236166f432696c72afb87ff0468 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "c3e9a236166f432696c72afb87ff0468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } health_report { overall_health: UNKNOWN } } }
W20260807 04:25:37.277323 16536 tablet.cc:2369] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:25:37.286623 16676 tablet.cc:2369] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:37.301362 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.295979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.1.244.195:33183)
0807 04:25:37.296179 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.296185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.296264 (+ 79us) server_negotiation.cc:407] Connection header received
0807 04:25:37.296513 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.296517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.296600 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.296748 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:25:37.297135 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.297840 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.299253 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.299668 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.299803 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.300336 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:25:37.300793 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:25:37.300944 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.301030 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.301106 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 04:25:37.302554 16818 tablet.cc:2369] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:37.308897 16488 raft_consensus.cc:1261] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Refusing update from remote peer c3e9a236166f432696c72afb87ff0468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:37.308933 16629 raft_consensus.cc:1261] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Refusing update from remote peer c3e9a236166f432696c72afb87ff0468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:37.309787 16841 consensus_queue.cc:1035] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:37.310072 16845 consensus_queue.cc:1035] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:37.321014 16849 mvcc.cc:205] Tried to move back new op lower bound from 7315770316007895040 to 7315770315852513280. Current Snapshot: MvccSnapshot[applied={T|T < 7315770316007895040}]
I20260807 04:25:37.334522 16333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:25:37.334793 16333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:25:37.338217 16333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Sending DeleteTablet for 3 replicas of tablet 0788826e596840269cb1418c3c6958d3
I20260807 04:25:37.339571 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16397
I20260807 04:25:37.339509 16747 tablet_service.cc:1514] Processing DeleteTablet for tablet 0788826e596840269cb1418c3c6958d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:37 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:42284
I20260807 04:25:37.340044 16608 tablet_service.cc:1514] Processing DeleteTablet for tablet 0788826e596840269cb1418c3c6958d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:37 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33662
I20260807 04:25:37.340402 16466 tablet_service.cc:1514] Processing DeleteTablet for tablet 0788826e596840269cb1418c3c6958d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:37 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:49768
I20260807 04:25:37.341418 16859 tablet_replica.cc:330] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: stopping tablet replica
I20260807 04:25:37.341969 16859 raft_consensus.cc:2229] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:25:37.342794 16859 raft_consensus.cc:2258] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:37.349363 16859 ts_tablet_manager.cc:1905] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:37.352893 16862 tablet_replica.cc:330] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: stopping tablet replica
I20260807 04:25:37.353307 16862 raft_consensus.cc:2229] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:25:37.353821 16862 raft_consensus.cc:2258] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:37.359829 16862 ts_tablet_manager.cc:1905] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:37.360073 16859 ts_tablet_manager.cc:1918] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:37.360288 16859 log.cc:1196] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/0788826e596840269cb1418c3c6958d3
I20260807 04:25:37.360976 16859 ts_tablet_manager.cc:1939] T 0788826e596840269cb1418c3c6958d3 P c3e9a236166f432696c72afb87ff0468: Deleting consensus metadata
I20260807 04:25:37.363169 16318 catalog_manager.cc:4841] TS c3e9a236166f432696c72afb87ff0468 (127.1.244.195:33183): tablet 0788826e596840269cb1418c3c6958d3 (table test-table [id=9bd39b471b0946ffbaca8a5169f0aa47]) successfully deleted
I20260807 04:25:37.366663 16862 ts_tablet_manager.cc:1918] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:37.366830 16862 log.cc:1196] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/0788826e596840269cb1418c3c6958d3
I20260807 04:25:37.367375 16862 ts_tablet_manager.cc:1939] T 0788826e596840269cb1418c3c6958d3 P f786f020d78446e9b0ac9729e25a7879: Deleting consensus metadata
I20260807 04:25:37.368898 16860 tablet_replica.cc:330] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: stopping tablet replica
I20260807 04:25:37.369168 16318 catalog_manager.cc:4841] TS f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215): tablet 0788826e596840269cb1418c3c6958d3 (table test-table [id=9bd39b471b0946ffbaca8a5169f0aa47]) successfully deleted
I20260807 04:25:37.369374 16860 raft_consensus.cc:2229] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 07 04:25:37 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:25:37.369930 16860 raft_consensus.cc:2258] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:37.372118 16860 ts_tablet_manager.cc:1905] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:37.378288 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.340805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.1.244.193:44215)
0807 04:25:37.341008 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.341016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.350152 (+ 9136us) server_negotiation.cc:407] Connection header received
0807 04:25:37.350335 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.350340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.350431 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.350523 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:37.354206 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.355372 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.357638 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.358292 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.358404 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.371018 (+ 12614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:37.371032 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:37.371051 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:37.371092 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:37.375016 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.375924 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.375937 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.375944 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.376135 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.376570 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.376575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.376579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.377005 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:37.377299 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.377705 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.377917 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:25:37.378784 16860 ts_tablet_manager.cc:1918] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:37.378964 16530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:37.378971 16860 log.cc:1196] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/0788826e596840269cb1418c3c6958d3
I20260807 04:25:37.379571 16860 ts_tablet_manager.cc:1939] T 0788826e596840269cb1418c3c6958d3 P 17f66e1481ad49dbab502ccadb055c21: Deleting consensus metadata
I20260807 04:25:37.382079 16319 catalog_manager.cc:4841] TS 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194:45085): tablet 0788826e596840269cb1418c3c6958d3 (table test-table [id=9bd39b471b0946ffbaca8a5169f0aa47]) successfully deleted
I20260807 04:25:37.502597 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16397
I20260807 04:25:37.529348 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16542
Aug 07 04:25:37 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:25:37.548878 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.530246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.1.244.194:45085)
0807 04:25:37.530411 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.530415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.530433 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:25:37.530590 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.530594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.530655 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.530728 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:37.532401 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.533476 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.535427 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.536128 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.536250 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.541644 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:37.541656 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:37.541672 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:37.541706 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:37.545795 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.546616 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.546625 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.546629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.546772 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.547206 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.547211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.547215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.547671 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:37.547974 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.548306 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.548532 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:25:37.549530 16670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:37.649006 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16542
I20260807 04:25:37.667974 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16680
Aug 07 04:25:37 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:25:37.691052 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:37.668812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.1.244.195:33183)
0807 04:25:37.668990 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:37.668995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:37.669053 (+ 58us) server_negotiation.cc:407] Connection header received
0807 04:25:37.669269 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:37.669273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:37.669332 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:37.669422 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:37.671331 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.672481 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.674472 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:37.674915 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:37.675002 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:37.684917 (+ 9915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:37.684952 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:37.684973 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:37.685042 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:37.688196 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.688887 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.688895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.688898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.689040 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:37.689427 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:37.689436 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:37.689438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:37.689795 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:37.690139 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:37.690427 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 04:25:37.690680 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:25:37.691807 16811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:37.816444 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16680
I20260807 04:25:37.841714 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16300
I20260807 04:25:37.843128 16365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:37.961279 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16300
I20260807 04:25:37.985450 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41841
--webserver_interface=127.1.244.254
--webserver_port=43239
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:38.317225 16867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:38.317502 16867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:38.317677 16867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:38.324894 16867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:38.325028 16867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:38.325133 16867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:38.325238 16867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:38.325335 16867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:38.335218 16867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=43239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:38.336565 16867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:38.337848 16867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:38.364594 16874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:38.364594 16877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:38.364627 16875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:38.365481 16867 server_base.cc:1034] running on GCE node
I20260807 04:25:38.366297 16867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:38.367990 16867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:38.369236 16867 hybrid_clock.cc:648] HybridClock initialized: now 1786076738369158 us; error 52 us; skew 500 ppm
I20260807 04:25:38.369647 16867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:38 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:38.373225 16867 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:38.374805 16867 webserver.cc:460] Webserver started at http://127.1.244.254:43239/ using document root <none> and password file <none>
I20260807 04:25:38.375556 16867 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:38.375712 16867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:38.381697 16867 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:25:38.384723 16885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:38.386116 16867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:25:38.386255 16867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "6a1776690e684ae9967c78c8ba3f5aad"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "9cecfa151235a6c57e501b93d773dc34"
server_key_iv: "edeb8e40bebb58a7c432ac80dbc6e51a"
server_key_version: "encryptionkey@0"
I20260807 04:25:38.387269 16867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:38.404667 16867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:38.408591 16867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:38.408810 16867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:38.427701 16867 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41841
I20260807 04:25:38.427712 16938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41841 every 8 connection(s)
I20260807 04:25:38.429248 16867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:38.436233 16939 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:25:38.438910 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16867
I20260807 04:25:38.439507 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap starting.
I20260807 04:25:38.439783 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:44215
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:41841
--webserver_port=35061
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:38.459183 16939 log.cc:824] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:38.471222 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:25:38.471757 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap complete.
I20260807 04:25:38.481022 16939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:38.481844 16939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a1776690e684ae9967c78c8ba3f5aad, State: Initialized, Role: FOLLOWER
I20260807 04:25:38.482425 16939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:38.482676 16939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:25:38.482818 16939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:25:38.483011 16939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:25:38.487043 16939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:38.487573 16939 leader_election.cc:302] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a1776690e684ae9967c78c8ba3f5aad; no voters:
I20260807 04:25:38.488204 16939 leader_election.cc:288] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:25:38.488415 16944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:25:38.489646 16944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 LEADER]: Becoming Leader. State: Replica: 6a1776690e684ae9967c78c8ba3f5aad, State: Running, Role: LEADER
I20260807 04:25:38.490383 16939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:38.490199 16944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:38.494393 16945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6a1776690e684ae9967c78c8ba3f5aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } } }
I20260807 04:25:38.494517 16946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a1776690e684ae9967c78c8ba3f5aad. Latest consensus state: current_term: 2 leader_uuid: "6a1776690e684ae9967c78c8ba3f5aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } } }
I20260807 04:25:38.494792 16945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:38.494928 16946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:38.509193 16954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:38.517095 16954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:38.518874 16954 catalog_manager.cc:1259] Loaded cluster ID: 8afed9de62584979b8115fd0c33ecf31
I20260807 04:25:38.519004 16954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:25:38.523167 16954 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 04:25:38.537549 16954 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 04:25:38.544608 16954 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:25:38.546098 16954 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Loaded TSK: 0
I20260807 04:25:38.547188 16954 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:25:38.807832 16942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:38.808175 16942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:38.808350 16942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:38.816121 16942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:38.816217 16942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:38.816372 16942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:25:38.825884 16942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:44215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=35061
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:38.827176 16942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:38.828562 16942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:38.843294 16969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:38.843362 16968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:38.843808 16971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:38.845081 16942 server_base.cc:1034] running on GCE node
I20260807 04:25:38.845745 16942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:38.846729 16942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:38.848013 16942 hybrid_clock.cc:648] HybridClock initialized: now 1786076738847958 us; error 47 us; skew 500 ppm
I20260807 04:25:38.848325 16942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:38 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:38.855355 16942 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:25:38.856890 16942 webserver.cc:460] Webserver started at http://127.1.244.193:35061/ using document root <none> and password file <none>
I20260807 04:25:38.857635 16942 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:38.857796 16942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:38.863132 16942 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 04:25:38.865900 16978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:38.867314 16942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:25:38.867472 16942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "f786f020d78446e9b0ac9729e25a7879"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "13e96597473fa95c8341637f880e0bcf"
server_key_iv: "aa47107da6252ef1a2b21c87afc1d4ca"
server_key_version: "encryptionkey@0"
I20260807 04:25:38.868508 16942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:38.884953 16942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:38.889348 16942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:38.889624 16942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:38.890762 16942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:38.892335 16942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:38.892444 16942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:38.892592 16942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:38.892686 16942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:38.926491 16942 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44215
I20260807 04:25:38.926584 17095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44215 every 8 connection(s)
I20260807 04:25:38.928012 16942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:25:38.936898 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 16942
I20260807 04:25:38.937547 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:45085
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:41841
--webserver_port=45949
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:25:38 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076738, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:38.950945 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:38.930846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:60093 (local address 127.1.244.254:41841)
0807 04:25:38.931514 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:38.931532 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:38.931943 (+ 411us) server_negotiation.cc:407] Connection header received
0807 04:25:38.933972 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:38.933984 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:38.934282 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:38.934565 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:38.935979 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:38.937275 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:38.938897 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:38.939530 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:38.939744 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:38.943119 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:38.943127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:38.943139 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:38.943179 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:38.945953 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:38.946898 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:38.946908 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:38.946917 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:38.947128 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:38.947648 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:38.947655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:38.947659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:38.948385 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:38.948572 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:38.949334 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 04:25:38.949698 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":244,"threads_started":1}
I20260807 04:25:38.950943 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:38.930959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:60093)
0807 04:25:38.931735 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:38.931787 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:38.933479 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:38.934684 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:38.934691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:38.935159 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:38.935705 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:38.935722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:38.937445 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:38.937450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:38.938739 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:38.938752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:38.939762 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:38.939769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:38.940075 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:38.940755 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:38.940776 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:38.942858 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:38.946089 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:38.946098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:38.946110 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:38.946739 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:38.947253 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:38.947256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:38.947258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:38.947489 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:38.948537 (+ 1048us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:38.948544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:38.948868 (+ 324us) client_negotiation.cc:769] Sending connection context
0807 04:25:38.949131 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 04:25:38.949645 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":310,"threads_started":1}
I20260807 04:25:38.954175 17096 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41841
I20260807 04:25:38.954495 17096 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:38.955354 17096 heartbeater.cc:511] Master 127.1.244.254:41841 requested a full tablet report, sending...
I20260807 04:25:38.957701 16903 ts_manager.cc:194] Registered new tserver with Master: f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215)
I20260807 04:25:38.959172 16903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:60093
W20260807 04:25:39.268826 17100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:39.269090 17100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:39.269296 17100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:39.276594 17100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:39.276710 17100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:39.276930 17100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:25:39.286408 17100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:45085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=45949
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:39.287599 17100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:39.288919 17100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:39.302273 17106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:39.302297 17109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:39.302982 17105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:39.303031 17100 server_base.cc:1034] running on GCE node
I20260807 04:25:39.303666 17100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:39.304694 17100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:39.305917 17100 hybrid_clock.cc:648] HybridClock initialized: now 1786076739305875 us; error 33 us; skew 500 ppm
I20260807 04:25:39.306227 17100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:39 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076739, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:39.309324 17100 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:25:39.310708 17100 webserver.cc:460] Webserver started at http://127.1.244.194:45949/ using document root <none> and password file <none>
I20260807 04:25:39.311367 17100 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:39.311532 17100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:39.316618 17100 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:25:39.319115 17116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:39.320379 17100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:25:39.320546 17100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "17f66e1481ad49dbab502ccadb055c21"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "fbf6a269dcd02e052d74a574a6917eeb"
server_key_iv: "893c96314ecc64d7bcb7ee382b7635f3"
server_key_version: "encryptionkey@0"
I20260807 04:25:39.321444 17100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:39.335752 17100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:39.339300 17100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:39.339613 17100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:39.340622 17100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:39.341866 17100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:39.341989 17100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:39.342114 17100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:39.342212 17100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:39.375968 17100 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45085
I20260807 04:25:39.376056 17233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45085 every 8 connection(s)
I20260807 04:25:39.377456 17100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:25:39.385624 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 17100
I20260807 04:25:39.386222 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:33183
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:41841
--webserver_port=42679
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:25:39 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076739, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:39.398491 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.379818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:39277 (local address 127.1.244.254:41841)
0807 04:25:39.380029 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.380034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.381128 (+ 1094us) server_negotiation.cc:407] Connection header received
0807 04:25:39.382926 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.382932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.383028 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.383139 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:39.384685 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.385596 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.387283 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.388096 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.388242 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.391709 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:39.391718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:39.391726 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:39.391786 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:39.395471 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:39.396206 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:39.396214 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:39.396220 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:39.396376 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:39.396833 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:39.396837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:39.396839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:39.397093 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:39.397202 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.398071 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.398240 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:25:39.399991 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.380255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:39277)
0807 04:25:39.380961 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:39.381004 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:39.382652 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:39.383378 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:39.383386 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:39.383861 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:39.384507 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.384525 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.385778 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.385787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.387119 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.387132 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.388291 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.388296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.388498 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.389169 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:39.389190 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:39.391478 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:39.395603 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:39.395610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:39.395621 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:39.396094 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:39.396494 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:39.396497 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:39.396499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:39.396708 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:39.397217 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:39.397227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:39.397662 (+ 435us) client_negotiation.cc:769] Sending connection context
0807 04:25:39.397984 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 04:25:39.398544 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":285,"threads_started":1}
I20260807 04:25:39.401518 17234 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41841
I20260807 04:25:39.401870 17234 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:39.402608 17234 heartbeater.cc:511] Master 127.1.244.254:41841 requested a full tablet report, sending...
I20260807 04:25:39.404104 16903 ts_manager.cc:194] Registered new tserver with Master: 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194:45085)
I20260807 04:25:39.404934 16903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:39277
W20260807 04:25:39.720793 17237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:39.721048 17237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:39.721215 17237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:39.728639 17237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:39.728731 17237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:39.728942 17237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:25:39.738453 17237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:33183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=42679
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:39.739653 17237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:39.740926 17237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:39.753698 17242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:39.753656 17243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:39.753731 17245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:39.754592 17237 server_base.cc:1034] running on GCE node
I20260807 04:25:39.755183 17237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:39.756096 17237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:39.757299 17237 hybrid_clock.cc:648] HybridClock initialized: now 1786076739757245 us; error 46 us; skew 500 ppm
I20260807 04:25:39.757596 17237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:39 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076739, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:39.760705 17237 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:25:39.762156 17237 webserver.cc:460] Webserver started at http://127.1.244.195:42679/ using document root <none> and password file <none>
I20260807 04:25:39.762796 17237 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:39.762955 17237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:39.768045 17237 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 04:25:39.770515 17252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:39.771816 17237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:25:39.772048 17237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "c3e9a236166f432696c72afb87ff0468"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "58da0a0202cf3a068cc4c68af86ead90"
server_key_iv: "6a9bda6cc97f706bdb4a65fe2e4445b9"
server_key_version: "encryptionkey@0"
I20260807 04:25:39.773145 17237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:39.806675 17237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:39.810607 17237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:39.810895 17237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:39.811911 17237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:39.813285 17237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:39.813416 17237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:39.813555 17237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:39.813644 17237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:39.846761 17237 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:33183
I20260807 04:25:39.846858 17369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:33183 every 8 connection(s)
I20260807 04:25:39.848351 17237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:25:39.856513 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 17237
Aug 07 04:25:39 dist-test-slave-zkpd krb5kdc[16287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076739, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:25:39.869536 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.850889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:51331 (local address 127.1.244.254:41841)
0807 04:25:39.851069 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.851073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.852232 (+ 1159us) server_negotiation.cc:407] Connection header received
0807 04:25:39.853998 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.854002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.854067 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.854153 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:39.855519 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.856385 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.858245 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.858910 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.859060 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.862706 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:39.862714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:39.862719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:39.862760 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:39.866406 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:39.867081 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:39.867086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:39.867089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:39.867254 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:39.867720 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:39.867723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:39.867725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:39.868042 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:39.868228 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.868869 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.869095 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:25:39.870625 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.851337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:51331)
0807 04:25:39.852078 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:39.852117 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:39.853752 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:39.854356 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:39.854364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:39.854849 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:25:39.855382 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.855398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.856563 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.856567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.858111 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.858123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.859145 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.859150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.859432 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.860143 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:25:39.860164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:25:39.862503 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:25:39.866541 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:39.866548 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:39.866559 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:39.866960 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:39.867374 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:25:39.867380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:25:39.867383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:25:39.867604 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:25:39.868203 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:25:39.868209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:25:39.868502 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 04:25:39.868724 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 04:25:39.869198 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":269,"threads_started":1}
I20260807 04:25:39.872116 17370 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41841
I20260807 04:25:39.872468 17370 heartbeater.cc:464] Registering TS with master...
I20260807 04:25:39.873349 17370 heartbeater.cc:511] Master 127.1.244.254:41841 requested a full tablet report, sending...
I20260807 04:25:39.873580 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.857414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54076 (local address 127.1.244.254:41841)
0807 04:25:39.857887 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.857892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.858295 (+ 403us) server_negotiation.cc:407] Connection header received
0807 04:25:39.858418 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.858421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.858471 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.858548 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:39.860566 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.861606 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.863195 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.863817 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.863968 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.867169 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:39.867173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:39.867176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:39.867205 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:39.870424 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:39.871185 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:39.871190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:39.871193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:39.871355 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:39.871772 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:39.871777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:39.871781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:39.872352 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:39.872629 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.872933 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.873155 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"spinlock_wait_cycles":9088,"thread_start_us":226,"threads_started":1}
I20260807 04:25:39.875005 16903 ts_manager.cc:194] Registered new tserver with Master: c3e9a236166f432696c72afb87ff0468 (127.1.244.195:33183)
I20260807 04:25:39.875857 16903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:51331
I20260807 04:25:39.876204 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:25:39.889524 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.877340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.1.244.254:41841)
0807 04:25:39.877534 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.877543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.883869 (+ 6326us) server_negotiation.cc:407] Connection header received
0807 04:25:39.884101 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.884109 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.884280 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.884429 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:25:39.884919 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.885912 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.887534 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.888018 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.888111 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.888614 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:25:39.889084 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:25:39.889184 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.889298 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.889355 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 04:25:39.890543 16903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:25:39.898021 16903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:25:39.931756 17382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.909558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:45085 (local address 127.0.0.1:33692)
0807 04:25:39.918671 (+ 9113us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:39.918695 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:39.918851 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:39.923250 (+ 4399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:39.923259 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:39.923314 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:39.923713 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.923726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.925973 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.925978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.928446 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.928461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.930766 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.930772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.930960 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.931017 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 04:25:39.931146 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 04:25:39.931352 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8975,"thread_start_us":8852,"threads_started":1}
I20260807 04:25:39.932752 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.912143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44215 (local address 127.0.0.1:49790)
0807 04:25:39.923250 (+ 11107us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:39.923267 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:39.923385 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:39.924399 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:39.924405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:39.924434 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:39.924771 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.924783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.926968 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.926973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.929217 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.929231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.931531 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.931538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.931715 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.931762 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 04:25:39.931864 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 04:25:39.932435 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10993,"thread_start_us":10870,"threads_started":1}
I20260807 04:25:39.932850 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.910210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.1.244.194:45085)
0807 04:25:39.922376 (+ 12166us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.922400 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.922438 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:25:39.922560 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.922572 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.922892 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.923144 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:39.923992 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.925799 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.928664 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.930258 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.930404 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.930609 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.932139 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.932357 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11944,"thread_start_us":11784,"threads_started":1}
I20260807 04:25:39.934007 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.912251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.1.244.193:44215)
0807 04:25:39.922825 (+ 10574us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.922839 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.923533 (+ 694us) server_negotiation.cc:407] Connection header received
0807 04:25:39.923642 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.923653 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.923927 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.924122 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:39.924986 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.926736 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.929463 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.930970 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.931134 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.931362 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.933422 (+ 2060us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.933586 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10331,"thread_start_us":11140,"threads_started":1}
I20260807 04:25:39.935039 17380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.908994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:33183 (local address 127.0.0.1:42308)
0807 04:25:39.911531 (+ 2537us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:39.911580 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:39.911764 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:39.924719 (+ 12955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:39.924724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:39.924772 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:39.925156 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.925165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.929515 (+ 4350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.929523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.931521 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:39.931535 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.934542 (+ 3007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.934546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:39.934662 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.934685 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:25:39.934733 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 04:25:39.934819 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2253,"thread_start_us":2096,"threads_started":1}
I20260807 04:25:39.935926 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:39.909271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42308 (local address 127.1.244.195:33183)
0807 04:25:39.924163 (+ 14892us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:39.924169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:39.924192 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:25:39.924262 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:39.924266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:39.924473 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:39.924722 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:39.925336 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.929344 (+ 4008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.931722 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:39.934297 (+ 2575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:39.935320 (+ 1023us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:39.935492 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:39.935543 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 04:25:39.935653 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14745,"thread_start_us":14563,"threads_started":1}
I20260807 04:25:39.936965 17165 tablet_service.cc:1467] Processing CreateTablet for tablet ca341d5bdbd1448da0da8bcef4c6de38 (DEFAULT_TABLE table=test-table [id=15f264fd600e42a49e069f955e6592c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:39.937707 17026 tablet_service.cc:1467] Processing CreateTablet for tablet ca341d5bdbd1448da0da8bcef4c6de38 (DEFAULT_TABLE table=test-table [id=15f264fd600e42a49e069f955e6592c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:39.938496 17165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca341d5bdbd1448da0da8bcef4c6de38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:39.939134 17026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca341d5bdbd1448da0da8bcef4c6de38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:39.940739 17303 tablet_service.cc:1467] Processing CreateTablet for tablet ca341d5bdbd1448da0da8bcef4c6de38 (DEFAULT_TABLE table=test-table [id=15f264fd600e42a49e069f955e6592c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:25:39.941901 17303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca341d5bdbd1448da0da8bcef4c6de38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:25:39.952869 17386 tablet_bootstrap.cc:492] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Bootstrap starting.
I20260807 04:25:39.955840 17386 tablet_bootstrap.cc:654] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:39.956679 17387 tablet_bootstrap.cc:492] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Bootstrap starting.
I20260807 04:25:39.957233 17386 log.cc:824] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:39.958236 17388 tablet_bootstrap.cc:492] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Bootstrap starting.
I20260807 04:25:39.960472 17386 tablet_bootstrap.cc:492] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: No bootstrap required, opened a new log
I20260807 04:25:39.960731 17386 ts_tablet_manager.cc:1397] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 04:25:39.960865 17387 tablet_bootstrap.cc:654] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:39.961735 17388 tablet_bootstrap.cc:654] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Neither blocks nor log segments found. Creating new log.
I20260807 04:25:39.962134 17387 log.cc:824] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:39.962993 17096 heartbeater.cc:503] Master 127.1.244.254:41841 was elected leader, sending a full tablet report...
I20260807 04:25:39.963966 17388 log.cc:824] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:39.965446 17387 tablet_bootstrap.cc:492] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: No bootstrap required, opened a new log
I20260807 04:25:39.965768 17387 ts_tablet_manager.cc:1397] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 04:25:39.969875 17388 tablet_bootstrap.cc:492] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: No bootstrap required, opened a new log
I20260807 04:25:39.970135 17388 ts_tablet_manager.cc:1397] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260807 04:25:39.971710 17386 raft_consensus.cc:348] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:39.972182 17386 raft_consensus.cc:374] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:39.972292 17386 raft_consensus.cc:729] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e9a236166f432696c72afb87ff0468, State: Initialized, Role: FOLLOWER
I20260807 04:25:39.973038 17386 consensus_queue.cc:260] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:39.974248 17370 heartbeater.cc:503] Master 127.1.244.254:41841 was elected leader, sending a full tablet report...
I20260807 04:25:39.974915 17386 ts_tablet_manager.cc:1428] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260807 04:25:39.975728 17387 raft_consensus.cc:348] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:39.976142 17387 raft_consensus.cc:374] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:39.976255 17387 raft_consensus.cc:729] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f786f020d78446e9b0ac9729e25a7879, State: Initialized, Role: FOLLOWER
I20260807 04:25:39.976904 17387 consensus_queue.cc:260] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:39.978610 17387 ts_tablet_manager.cc:1428] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Time spent starting tablet: real 0.013s user 0.010s sys 0.000s
I20260807 04:25:39.980253 17388 raft_consensus.cc:348] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:39.980609 17388 raft_consensus.cc:374] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:25:39.980718 17388 raft_consensus.cc:729] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f66e1481ad49dbab502ccadb055c21, State: Initialized, Role: FOLLOWER
I20260807 04:25:39.981256 17388 consensus_queue.cc:260] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:39.982218 17234 heartbeater.cc:503] Master 127.1.244.254:41841 was elected leader, sending a full tablet report...
I20260807 04:25:39.982583 17388 ts_tablet_manager.cc:1428] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 04:25:40.052017 17394 raft_consensus.cc:482] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:25:40.052364 17394 raft_consensus.cc:504] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:40.054119 17394 leader_election.cc:288] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3e9a236166f432696c72afb87ff0468 (127.1.244.195:33183), f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215)
I20260807 04:25:40.061439 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.054625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:46961 (local address 127.1.244.195:33183)
0807 04:25:40.054825 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:40.054829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:40.055411 (+ 582us) server_negotiation.cc:407] Connection header received
0807 04:25:40.055679 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:40.055688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:40.055784 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:40.055847 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:40.056581 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.057680 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.059538 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.060483 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.060662 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.060744 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:40.061046 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 04:25:40.061173 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:25:40.061618 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.054522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:33183 (local address 127.1.244.194:46961)
0807 04:25:40.055303 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:40.055325 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:40.055469 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:40.056082 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:40.056085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:40.056107 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:40.056444 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:40.056452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.057808 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.057812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:40.059369 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:40.059385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.060706 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.060712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:40.060860 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.060896 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:25:40.061018 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 04:25:40.061147 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":65,"spinlock_wait_cycles":16256,"thread_start_us":455,"threads_started":1}
I20260807 04:25:40.061632 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.055653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:35099 (local address 127.1.244.193:44215)
0807 04:25:40.055866 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:40.055873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:40.056174 (+ 301us) server_negotiation.cc:407] Connection header received
0807 04:25:40.056421 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:40.056426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:40.056488 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:40.056596 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:25:40.057065 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.058003 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.059934 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.060688 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.060875 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.060962 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:40.061245 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 04:25:40.061377 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:25:40.061786 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.055458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:44215 (local address 127.1.244.194:35099)
0807 04:25:40.056061 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:40.056075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:25:40.056255 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:25:40.056607 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:25:40.056610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:25:40.056627 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:25:40.056917 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:40.056924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.058145 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.058151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:40.059742 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:25:40.059755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.060904 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.060910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:25:40.061086 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.061110 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:25:40.061215 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 04:25:40.061339 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":3,"thread_start_us":283,"threads_started":1}
I20260807 04:25:40.062852 17047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca341d5bdbd1448da0da8bcef4c6de38" candidate_uuid: "17f66e1481ad49dbab502ccadb055c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f786f020d78446e9b0ac9729e25a7879" is_pre_election: true
I20260807 04:25:40.063136 17324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca341d5bdbd1448da0da8bcef4c6de38" candidate_uuid: "17f66e1481ad49dbab502ccadb055c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3e9a236166f432696c72afb87ff0468" is_pre_election: true
I20260807 04:25:40.063416 17047 raft_consensus.cc:2454] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f66e1481ad49dbab502ccadb055c21 in term 0.
I20260807 04:25:40.063684 17324 raft_consensus.cc:2454] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f66e1481ad49dbab502ccadb055c21 in term 0.
I20260807 04:25:40.064181 17119 leader_election.cc:302] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f66e1481ad49dbab502ccadb055c21, f786f020d78446e9b0ac9729e25a7879; no voters:
I20260807 04:25:40.064594 17394 raft_consensus.cc:2781] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:25:40.064743 17394 raft_consensus.cc:482] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:25:40.064860 17394 raft_consensus.cc:3037] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:40.067057 17394 raft_consensus.cc:504] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:40.067647 17394 leader_election.cc:288] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [CANDIDATE]: Term 1 election: Requested vote from peers c3e9a236166f432696c72afb87ff0468 (127.1.244.195:33183), f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215)
I20260807 04:25:40.068112 17324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca341d5bdbd1448da0da8bcef4c6de38" candidate_uuid: "17f66e1481ad49dbab502ccadb055c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3e9a236166f432696c72afb87ff0468"
I20260807 04:25:40.068254 17047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca341d5bdbd1448da0da8bcef4c6de38" candidate_uuid: "17f66e1481ad49dbab502ccadb055c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f786f020d78446e9b0ac9729e25a7879"
I20260807 04:25:40.068334 17324 raft_consensus.cc:3037] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:40.068487 17047 raft_consensus.cc:3037] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:25:40.070575 17324 raft_consensus.cc:2454] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f66e1481ad49dbab502ccadb055c21 in term 1.
I20260807 04:25:40.070680 17047 raft_consensus.cc:2454] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f66e1481ad49dbab502ccadb055c21 in term 1.
I20260807 04:25:40.070981 17119 leader_election.cc:302] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f66e1481ad49dbab502ccadb055c21, c3e9a236166f432696c72afb87ff0468; no voters:
I20260807 04:25:40.071280 17394 raft_consensus.cc:2781] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:25:40.071789 17394 raft_consensus.cc:686] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 1 LEADER]: Becoming Leader. State: Replica: 17f66e1481ad49dbab502ccadb055c21, State: Running, Role: LEADER
I20260807 04:25:40.072330 17394 consensus_queue.cc:237] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } }
I20260807 04:25:40.077129 16902 catalog_manager.cc:5495] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "17f66e1481ad49dbab502ccadb055c21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f66e1481ad49dbab502ccadb055c21" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 45085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 } health_report { overall_health: UNKNOWN } } }
W20260807 04:25:40.100009 17371 tablet.cc:2369] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:40.112798 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.107315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.1.244.194:45085)
0807 04:25:40.107523 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:40.107529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:40.107622 (+ 93us) server_negotiation.cc:407] Connection header received
0807 04:25:40.107855 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:40.107867 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:40.107963 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:40.108113 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:25:40.108493 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.109223 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.110655 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.111059 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.111191 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.111717 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:25:40.112223 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:25:40.112382 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:40.112448 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 04:25:40.112526 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:25:40.120473 17047 raft_consensus.cc:1261] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Refusing update from remote peer 17f66e1481ad49dbab502ccadb055c21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:40.120517 17324 raft_consensus.cc:1261] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Refusing update from remote peer 17f66e1481ad49dbab502ccadb055c21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:25:40.121436 17397 consensus_queue.cc:1035] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3e9a236166f432696c72afb87ff0468" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 33183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:25:40.121780 17394 consensus_queue.cc:1035] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f786f020d78446e9b0ac9729e25a7879" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 44215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:25:40.129439 17235 tablet.cc:2369] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:25:40.152793 16903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:25:40.153126 16903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:25:40.156992 16903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Sending DeleteTablet for 3 replicas of tablet ca341d5bdbd1448da0da8bcef4c6de38
I20260807 04:25:40.158211 17303 tablet_service.cc:1514] Processing DeleteTablet for tablet ca341d5bdbd1448da0da8bcef4c6de38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:40 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:42308
I20260807 04:25:40.158401 17165 tablet_service.cc:1514] Processing DeleteTablet for tablet ca341d5bdbd1448da0da8bcef4c6de38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:40 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:33692
I20260807 04:25:40.158890 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16942
I20260807 04:25:40.159406 17026 tablet_service.cc:1514] Processing DeleteTablet for tablet ca341d5bdbd1448da0da8bcef4c6de38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:25:40 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:49790
I20260807 04:25:40.160152 17412 tablet_replica.cc:330] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: stopping tablet replica
I20260807 04:25:40.160614 17412 raft_consensus.cc:2229] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:25:40.161302 17412 raft_consensus.cc:2258] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:40.167969 17412 ts_tablet_manager.cc:1905] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:40.168629 17413 tablet_replica.cc:330] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: stopping tablet replica
I20260807 04:25:40.169098 17413 raft_consensus.cc:2229] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:25:40.169631 17413 raft_consensus.cc:2258] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:40.170464 17411 tablet_replica.cc:330] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: stopping tablet replica
I20260807 04:25:40.170867 17411 raft_consensus.cc:2229] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:25:40.171356 17411 raft_consensus.cc:2258] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:25:40.171619 17413 ts_tablet_manager.cc:1905] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:40.173509 17411 ts_tablet_manager.cc:1905] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:25:40.179080 17413 ts_tablet_manager.cc:1918] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:40.179255 17413 log.cc:1196] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/ca341d5bdbd1448da0da8bcef4c6de38
I20260807 04:25:40.179843 17413 ts_tablet_manager.cc:1939] T ca341d5bdbd1448da0da8bcef4c6de38 P f786f020d78446e9b0ac9729e25a7879: Deleting consensus metadata
I20260807 04:25:40.180371 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.160069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.1.244.193:44215)
0807 04:25:40.160516 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:40.160525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:40.160543 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:25:40.160601 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:40.160609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:40.160693 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:40.160795 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:40.162914 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.164221 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.166231 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.166866 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.167001 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.171300 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:40.171312 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:40.171327 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:40.171368 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:40.175356 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:40.176439 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:40.176448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:40.176452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:40.176608 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:40.177105 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:40.177109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:40.177112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:40.177717 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:40.178058 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:40.179561 (+ 1503us) server_negotiation.cc:299] Negotiation successful
0807 04:25:40.179820 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260807 04:25:40.180481 17411 ts_tablet_manager.cc:1918] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:40.180634 17411 log.cc:1196] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/ca341d5bdbd1448da0da8bcef4c6de38
I20260807 04:25:40.181214 17411 ts_tablet_manager.cc:1939] T ca341d5bdbd1448da0da8bcef4c6de38 P c3e9a236166f432696c72afb87ff0468: Deleting consensus metadata
I20260807 04:25:40.181391 17091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:40.181872 16888 catalog_manager.cc:4841] TS f786f020d78446e9b0ac9729e25a7879 (127.1.244.193:44215): tablet ca341d5bdbd1448da0da8bcef4c6de38 (table test-table [id=15f264fd600e42a49e069f955e6592c2]) successfully deleted
I20260807 04:25:40.183228 16888 catalog_manager.cc:4841] TS c3e9a236166f432696c72afb87ff0468 (127.1.244.195:33183): tablet ca341d5bdbd1448da0da8bcef4c6de38 (table test-table [id=15f264fd600e42a49e069f955e6592c2]) successfully deleted
I20260807 04:25:40.189273 17412 ts_tablet_manager.cc:1918] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:25:40.189495 17412 log.cc:1196] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/ca341d5bdbd1448da0da8bcef4c6de38
I20260807 04:25:40.190110 17412 ts_tablet_manager.cc:1939] T ca341d5bdbd1448da0da8bcef4c6de38 P 17f66e1481ad49dbab502ccadb055c21: Deleting consensus metadata
I20260807 04:25:40.192020 16889 catalog_manager.cc:4841] TS 17f66e1481ad49dbab502ccadb055c21 (127.1.244.194:45085): tablet ca341d5bdbd1448da0da8bcef4c6de38 (table test-table [id=15f264fd600e42a49e069f955e6592c2]) successfully deleted
I20260807 04:25:40.272819 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16942
I20260807 04:25:40.299984 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 17100
I20260807 04:25:40.314015 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.300829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.1.244.194:45085)
0807 04:25:40.301074 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:40.301080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:40.301099 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:25:40.301284 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:40.301290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:40.301380 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:40.301487 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:40.302833 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.303633 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.304992 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.305450 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.305579 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.308208 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:40.308222 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:40.308236 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:40.308277 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:40.311204 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:40.311873 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:40.311882 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:40.311886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:40.312086 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:40.312467 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:40.312473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:40.312477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:40.312821 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:40.313152 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:40.313415 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 04:25:40.313644 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 04:25:40.314841 17229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:40.426043 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 17100
I20260807 04:25:40.450903 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 17237
I20260807 04:25:40.464083 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:40.451700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.1.244.195:33183)
0807 04:25:40.451886 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:40.451940 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:40.451966 (+ 26us) server_negotiation.cc:407] Connection header received
0807 04:25:40.452094 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:40.452097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:40.452176 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:40.452260 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:40.453589 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.454321 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.455531 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:40.456022 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:40.456117 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:40.458643 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:40.458657 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:40.458674 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:40.458705 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:40.461679 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:40.462309 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:40.462315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:40.462317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:40.462455 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:40.462750 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:40.462753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:40.462755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:40.463076 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:40.463286 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:40.463536 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 04:25:40.463699 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:25:40.464735 17365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:40.552950 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 17237
I20260807 04:25:40.571623 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 16867
I20260807 04:25:40.572842 16934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:40.685241 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 16867
I20260807 04:25:40.710197 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41841
--webserver_interface=127.1.244.254
--webserver_port=43239
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:25:41.044127 17418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:41.044409 17418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:41.044584 17418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:41.051647 17418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:25:41.051741 17418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:41.051786 17418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:41.051873 17418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:25:41.051954 17418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:25:41.061443 17418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=43239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:41.062793 17418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:41.064091 17418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:41.075881 17424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:41.075997 17426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:41.076237 17423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:41.076818 17418 server_base.cc:1034] running on GCE node
I20260807 04:25:41.077554 17418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:41.078827 17418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:41.079998 17418 hybrid_clock.cc:648] HybridClock initialized: now 1786076741079959 us; error 50 us; skew 500 ppm
I20260807 04:25:41.080322 17418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:41 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076741, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:41.083245 17418 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:25:41.084791 17418 webserver.cc:460] Webserver started at http://127.1.244.254:43239/ using document root <none> and password file <none>
I20260807 04:25:41.085456 17418 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:41.085599 17418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:41.090710 17418 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 04:25:41.093447 17434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:41.094694 17418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:25:41.094888 17418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "6a1776690e684ae9967c78c8ba3f5aad"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "9cecfa151235a6c57e501b93d773dc34"
server_key_iv: "edeb8e40bebb58a7c432ac80dbc6e51a"
server_key_version: "encryptionkey@0"
I20260807 04:25:41.095930 17418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:41.113142 17418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:41.116961 17418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:41.117234 17418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:41.136104 17418 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41841
I20260807 04:25:41.136137 17487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41841 every 8 connection(s)
I20260807 04:25:41.137588 17418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:25:41.144235 17488 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:25:41.146327 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 17418
I20260807 04:25:41.147372 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:44215
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:41841
--webserver_port=35061
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:41.148169 17488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap starting.
I20260807 04:25:41.171757 17488 log.cc:824] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Log is configured to *not* fsync() on all Append() calls
I20260807 04:25:41.187849 17488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:25:41.188449 17488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad: Bootstrap complete.
I20260807 04:25:41.197551 17488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:41.198411 17488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a1776690e684ae9967c78c8ba3f5aad, State: Initialized, Role: FOLLOWER
I20260807 04:25:41.199092 17488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:41.199333 17488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:25:41.199491 17488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:25:41.199681 17488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 2 FOLLOWER]: Advancing to term 3
I20260807 04:25:41.203464 17488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:41.204021 17488 leader_election.cc:302] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a1776690e684ae9967c78c8ba3f5aad; no voters:
I20260807 04:25:41.204700 17488 leader_election.cc:288] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 04:25:41.204838 17492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 3 FOLLOWER]: Leader election won for term 3
I20260807 04:25:41.206116 17492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [term 3 LEADER]: Becoming Leader. State: Replica: 6a1776690e684ae9967c78c8ba3f5aad, State: Running, Role: LEADER
I20260807 04:25:41.206737 17492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } }
I20260807 04:25:41.206971 17488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:25:41.209223 17495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a1776690e684ae9967c78c8ba3f5aad. Latest consensus state: current_term: 3 leader_uuid: "6a1776690e684ae9967c78c8ba3f5aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } } }
I20260807 04:25:41.209583 17495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:41.209972 17493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6a1776690e684ae9967c78c8ba3f5aad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1776690e684ae9967c78c8ba3f5aad" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41841 } } }
I20260807 04:25:41.210227 17493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1776690e684ae9967c78c8ba3f5aad [sys.catalog]: This master's current role is: LEADER
I20260807 04:25:41.217530 17500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:25:41.228621 17500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:25:41.230628 17500 catalog_manager.cc:1259] Loaded cluster ID: 8afed9de62584979b8115fd0c33ecf31
I20260807 04:25:41.230806 17500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 04:25:41.233492 17500 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f417a32b9ad google::LogMessage::Fail() at ??:0
@ 0x7f417a32e88a google::LogMessage::SendToLog() at ??:0
I20260807 04:25:41.240520 17510 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f417a32b4de google::LogMessage::Flush() at ??:0
@ 0x7f417a32ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f418fd62512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f418fd5f976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f418fde7610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f418fde7439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f417cf77345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f417cf7a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f417cf7a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f417cf55eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f418022f6db start_thread at ??:0
@ 0x7f41791c561f clone at ??:0
W20260807 04:25:41.523412 17490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:41.523733 17490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:41.523942 17490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:41.532090 17490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:41.532235 17490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:41.532487 17490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:25:41.542243 17490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:44215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=35061
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:41.543521 17490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:41.544902 17490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:41.559199 17530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:41.559294 17531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:41.559252 17534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:41.560257 17490 server_base.cc:1034] running on GCE node
I20260807 04:25:41.560849 17490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:41.561858 17490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:41.563060 17490 hybrid_clock.cc:648] HybridClock initialized: now 1786076741563023 us; error 38 us; skew 500 ppm
I20260807 04:25:41.563387 17490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:41 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076741, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:41.566720 17490 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:25:41.568326 17490 webserver.cc:460] Webserver started at http://127.1.244.193:35061/ using document root <none> and password file <none>
I20260807 04:25:41.569128 17490 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:41.569303 17490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:41.574659 17490 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 04:25:41.577344 17541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:41.578598 17490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:25:41.578778 17490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "f786f020d78446e9b0ac9729e25a7879"
format_stamp: "Formatted at 2026-08-07 04:25:35 on dist-test-slave-zkpd"
server_key: "13e96597473fa95c8341637f880e0bcf"
server_key_iv: "aa47107da6252ef1a2b21c87afc1d4ca"
server_key_version: "encryptionkey@0"
I20260807 04:25:41.579771 17490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:41.593534 17490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:41.597258 17490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:41.597532 17490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:41.598464 17490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:41.599768 17490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:41.599936 17490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:41.600069 17490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:41.600160 17490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:41.635020 17490 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:44215
I20260807 04:25:41.635105 17658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:44215 every 8 connection(s)
I20260807 04:25:41.636725 17490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:25:41.637336 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 17490
I20260807 04:25:41.638180 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:45085
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:41841
--webserver_port=45949
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:41.641945 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:41.640121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:36175)
0807 04:25:41.640874 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:41.641091 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":305,"threads_started":1}
W20260807 04:25:41.642632 17659 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:41841 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:41841: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
I20260807 04:25:41.643496 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:41.643167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:49751)
0807 04:25:41.643303 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:41.643365 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 04:25:41.644347 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:41.644037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:40611)
0807 04:25:41.644168 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:41.644245 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 04:25:41.644570 17659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:25:41.967767 17662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:41.968088 17662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:41.968271 17662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:41.975179 17662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:41.975302 17662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:41.975466 17662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:25:41.984745 17662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:45085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=45949
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:41.985877 17662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:41.987089 17662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:41.999378 17667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:41.999459 17668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:41.999428 17670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:42.000524 17662 server_base.cc:1034] running on GCE node
I20260807 04:25:42.001078 17662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:42.001914 17662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:42.003108 17662 hybrid_clock.cc:648] HybridClock initialized: now 1786076742003066 us; error 28 us; skew 500 ppm
I20260807 04:25:42.003413 17662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:42 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076742, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:42.006506 17662 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:25:42.007959 17662 webserver.cc:460] Webserver started at http://127.1.244.194:45949/ using document root <none> and password file <none>
I20260807 04:25:42.008608 17662 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:42.008738 17662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:42.013753 17662 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260807 04:25:42.016324 17678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:42.017565 17662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:25:42.017776 17662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "17f66e1481ad49dbab502ccadb055c21"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "fbf6a269dcd02e052d74a574a6917eeb"
server_key_iv: "893c96314ecc64d7bcb7ee382b7635f3"
server_key_version: "encryptionkey@0"
I20260807 04:25:42.019135 17662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:42.039740 17662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:42.043579 17662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:42.043941 17662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:42.045074 17662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:42.046345 17662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:42.046468 17662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:42.046597 17662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:42.046687 17662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:42.080065 17662 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:45085
I20260807 04:25:42.080157 17794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:45085 every 8 connection(s)
I20260807 04:25:42.081475 17662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:25:42.085186 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 17662
I20260807 04:25:42.085908 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:33183
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:41841
--webserver_port=42679
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:25:42.086475 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.084532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:34819)
0807 04:25:42.085369 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.085599 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":326,"threads_started":1}
W20260807 04:25:42.087141 17795 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:41841 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:41841: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
I20260807 04:25:42.088009 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.087659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:43545)
0807 04:25:42.087789 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.087850 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 04:25:42.088929 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.088514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:58311)
0807 04:25:42.088673 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.088769 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 04:25:42.089174 17795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:25:42.414179 17798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:25:42.414485 17798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:25:42.414644 17798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:25:42.421765 17798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:25:42.421923 17798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:25:42.422120 17798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:25:42.431468 17798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:33183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=42679
--tserver_master_addrs=127.1.244.254:41841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:25:42.432763 17798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:25:42.434094 17798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:25:42.447408 17803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:42.447459 17807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:25:42.447458 17804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:25:42.448302 17798 server_base.cc:1034] running on GCE node
I20260807 04:25:42.448858 17798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:25:42.449735 17798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:25:42.450968 17798 hybrid_clock.cc:648] HybridClock initialized: now 1786076742450918 us; error 27 us; skew 500 ppm
I20260807 04:25:42.451326 17798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:25:42 dist-test-slave-zkpd krb5kdc[16287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076742, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:25:42.454362 17798 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:25:42.455742 17798 webserver.cc:460] Webserver started at http://127.1.244.195:42679/ using document root <none> and password file <none>
I20260807 04:25:42.456460 17798 fs_manager.cc:359] Metadata directory not provided
I20260807 04:25:42.456600 17798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:25:42.461701 17798 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:25:42.464319 17814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:42.465634 17798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 04:25:42.465798 17798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "c3e9a236166f432696c72afb87ff0468"
format_stamp: "Formatted at 2026-08-07 04:25:36 on dist-test-slave-zkpd"
server_key: "58da0a0202cf3a068cc4c68af86ead90"
server_key_iv: "6a9bda6cc97f706bdb4a65fe2e4445b9"
server_key_version: "encryptionkey@0"
I20260807 04:25:42.466765 17798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:25:42.481485 17798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:25:42.485458 17798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:25:42.485708 17798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:25:42.486600 17798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:25:42.487818 17798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:25:42.487983 17798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:42.488103 17798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:25:42.488191 17798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:25:42.521130 17798 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:33183
I20260807 04:25:42.521212 17929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:33183 every 8 connection(s)
I20260807 04:25:42.522653 17798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:25:42.523078 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 17798
I20260807 04:25:42.527398 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.525711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:49453)
0807 04:25:42.526377 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.526579 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":265,"threads_started":1}
W20260807 04:25:42.528139 17930 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:41841 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:41841: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
I20260807 04:25:42.528932 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.528619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:42293)
0807 04:25:42.528746 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.528809 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 04:25:42.529690 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.529421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:60929)
0807 04:25:42.529528 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.529594 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 04:25:42.529910 17930 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 04:25:42.645943 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:42.645061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:43721)
0807 04:25:42.645570 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:42.645716 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":277,"threads_started":1}
I20260807 04:25:43.090484 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:43.089709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:48705)
0807 04:25:43.090176 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:43.090260 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
I20260807 04:25:43.531061 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:43.530351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:45797)
0807 04:25:43.530748 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:43.530842 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":195,"threads_started":1}
I20260807 04:25:43.647553 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:43.646865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:42703)
0807 04:25:43.647314 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:43.647393 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":211,"threads_started":1}
I20260807 04:25:44.092133 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:44.091353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:48179)
0807 04:25:44.091794 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:44.091888 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260807 04:25:44.532845 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:44.532107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:43167)
0807 04:25:44.532618 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:44.532695 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":279,"threads_started":1}
I20260807 04:25:44.649384 17941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:44.648495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:50875)
0807 04:25:44.649060 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:44.649160 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":323,"threads_started":1}
I20260807 04:25:45.093604 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:45.092920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:39263)
0807 04:25:45.093352 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:45.093432 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":244,"threads_started":1}
I20260807 04:25:45.534394 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:45.533758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:39045)
0807 04:25:45.534166 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:45.534238 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":217,"threads_started":1}
I20260807 04:25:45.651003 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:45.650319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:51141)
0807 04:25:45.650727 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:45.650803 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":205,"threads_started":1}
I20260807 04:25:46.095165 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:46.094431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:39899)
0807 04:25:46.094871 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:46.094959 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":213,"threads_started":1}
I20260807 04:25:46.536041 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:46.535177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:36435)
0807 04:25:46.535669 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:46.535756 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":248,"threads_started":1}
I20260807 04:25:46.652709 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:46.652028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:59029)
0807 04:25:46.652471 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:46.652548 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":220,"threads_started":1}
I20260807 04:25:47.096961 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:47.096268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:47389)
0807 04:25:47.096721 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:47.096800 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":235,"threads_started":1}
I20260807 04:25:47.537703 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:47.536854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:53951)
0807 04:25:47.537392 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:47.537506 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":303,"threads_started":1}
I20260807 04:25:47.654167 17950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:47.653492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:55035)
0807 04:25:47.653923 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:47.654011 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":227,"threads_started":1}
I20260807 04:25:48.098515 17951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:48.097818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:53445)
0807 04:25:48.098237 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:48.098331 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":213,"threads_started":1}
I20260807 04:25:48.539325 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:48.538519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:52783)
0807 04:25:48.539077 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:48.539168 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":335,"threads_started":1}
I20260807 04:25:48.655771 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:48.655014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:55889)
0807 04:25:48.655484 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:48.655562 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":203,"threads_started":1}
I20260807 04:25:49.100298 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:49.099381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:37967)
0807 04:25:49.099992 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:49.100078 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":240,"threads_started":1}
I20260807 04:25:49.541050 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:49.540306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:60671)
0807 04:25:49.540776 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:49.540850 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":213,"threads_started":1}
I20260807 04:25:49.657260 17956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:49.656593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:42559)
0807 04:25:49.657013 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:49.657095 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":220,"threads_started":1}
I20260807 04:25:50.102136 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:50.101202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:37519)
0807 04:25:50.101819 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:50.101932 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":262,"threads_started":1}
I20260807 04:25:50.542567 17958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:50.541895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:45491)
0807 04:25:50.542316 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:50.542404 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":211,"threads_started":1}
I20260807 04:25:50.658847 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:50.658154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:54421)
0807 04:25:50.658599 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:50.658678 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":222,"threads_started":1}
I20260807 04:25:51.103811 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:51.103068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:52995)
0807 04:25:51.103548 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:51.103631 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":241,"threads_started":1}
I20260807 04:25:51.544142 17961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:51.543382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:34469)
0807 04:25:51.543831 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:51.543962 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":223,"threads_started":1}
I20260807 04:25:51.660662 17962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:51.660038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:43809)
0807 04:25:51.660435 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:51.660511 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":208,"threads_started":1}
I20260807 04:25:52.105423 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:52.104721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:51923)
0807 04:25:52.105154 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:52.105249 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":232,"threads_started":1}
I20260807 04:25:52.545665 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:52.545009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:33205)
0807 04:25:52.545441 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:52.545516 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":233,"threads_started":1}
I20260807 04:25:52.662415 17966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:52.661675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:56579)
0807 04:25:52.662135 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:52.662223 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
I20260807 04:25:53.107173 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:53.106397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:44517)
0807 04:25:53.106895 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:53.107002 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":284,"threads_started":1}
I20260807 04:25:53.547384 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:53.546678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:60861)
0807 04:25:53.547113 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:53.547199 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":234,"threads_started":1}
I20260807 04:25:53.664171 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:53.663416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:50983)
0807 04:25:53.663840 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:53.663954 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":222,"threads_started":1}
I20260807 04:25:54.108748 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:54.108060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:60209)
0807 04:25:54.108487 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:54.108562 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":225,"threads_started":1}
I20260807 04:25:54.549068 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:54.548390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:41137)
0807 04:25:54.548830 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:54.548908 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260807 04:25:54.665825 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:54.665032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:38351)
0807 04:25:54.665546 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:54.665638 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":262,"threads_started":1}
I20260807 04:25:55.110520 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:55.109670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:54259)
0807 04:25:55.110221 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:55.110318 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":318,"threads_started":1}
I20260807 04:25:55.550662 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:55.549932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:59473)
0807 04:25:55.550394 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:55.550485 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
I20260807 04:25:55.667482 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:55.666811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:44679)
0807 04:25:55.667222 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:55.667300 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":215,"threads_started":1}
I20260807 04:25:56.112218 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:56.111405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:52715)
0807 04:25:56.111940 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:56.112032 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":272,"threads_started":1}
I20260807 04:25:56.552386 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:56.551683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:41121)
0807 04:25:56.552144 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:56.552217 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":271,"threads_started":1}
I20260807 04:25:56.669291 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:56.668648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:47405)
0807 04:25:56.669037 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:56.669119 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":205,"threads_started":1}
I20260807 04:25:57.114073 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:57.113326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:47215)
0807 04:25:57.113768 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:57.113877 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":229,"threads_started":1}
I20260807 04:25:57.536194 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 17490
I20260807 04:25:57.554315 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:57.553604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.195:57007)
0807 04:25:57.554030 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:57.554128 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":223,"threads_started":1}
I20260807 04:25:57.671437 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:57.670458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.193:52517)
0807 04:25:57.671047 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:57.671231 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":257,"threads_started":1}
I20260807 04:25:58.115978 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:58.115118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41841 (local address 127.1.244.194:38863)
0807 04:25:58.115660 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 04:25:58.115761 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:41841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":288,"threads_started":1}
I20260807 04:25:58.289191 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:57.537617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.1.244.193:44215)
0807 04:25:57.538355 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:57.538379 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:58.272618 (+734239us) server_negotiation.cc:407] Connection header received
0807 04:25:58.274355 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:58.274370 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:58.274879 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:58.275187 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:58.276608 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:58.277831 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:58.279169 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:58.279809 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:58.280020 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:58.282690 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:58.282698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:58.282712 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:58.282776 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:58.285864 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:58.286614 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:58.286620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:58.286626 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:58.286782 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:58.287195 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:58.287200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:58.287204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:58.287699 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:58.288042 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:58.288484 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 04:25:58.288884 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":359,"threads_started":1}
I20260807 04:25:58.290549 17654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:58.375735 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 17490
I20260807 04:25:58.397013 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 17662
I20260807 04:25:58.415079 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:58.398202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.1.244.194:45085)
0807 04:25:58.398857 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:58.398871 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:58.398922 (+ 51us) server_negotiation.cc:407] Connection header received
0807 04:25:58.400772 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:58.400781 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:58.401273 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:58.401560 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:58.402831 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:58.404353 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:58.405613 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:58.406245 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:58.406370 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:58.409025 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:58.409033 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:58.409046 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:58.409087 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:58.412140 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:58.412774 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:58.412780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:58.412785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:58.412948 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:58.413274 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:58.413276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:58.413278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:58.413778 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:58.414014 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:58.414361 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 04:25:58.414766 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":322,"threads_started":1}
I20260807 04:25:58.416503 17790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:58.500250 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 17662
I20260807 04:25:58.520165 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 17798
I20260807 04:25:58.538220 17988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:25:58.521150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.1.244.195:33183)
0807 04:25:58.521656 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 04:25:58.521671 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:25:58.521707 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:25:58.523408 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:25:58.523421 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:25:58.523851 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:25:58.524281 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:25:58.525819 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:58.527062 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:58.528368 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:25:58.529020 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:25:58.529219 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:25:58.532064 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:25:58.532073 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:25:58.532086 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:25:58.532124 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:25:58.534817 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:58.535575 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:58.535582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:58.535601 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:58.535779 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:25:58.536204 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:25:58.536209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:25:58.536212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:25:58.536717 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:25:58.537057 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 04:25:58.537508 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 04:25:58.537944 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":217,"threads_started":1}
I20260807 04:25:58.539529 17925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:25:58.622273 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 17798
2026-08-07T04:25:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25063 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:25:58 dist-test-slave-zkpd krb5kdc[17994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:25:58 dist-test-slave-zkpd krb5kdc[17994](info): set up 2 sockets
Aug 07 04:25:58 dist-test-slave-zkpd krb5kdc[17994](info): commencing operation
krb5kdc: starting...
W20260807 04:25:59.751021 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:25:59 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:25:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:25:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:26:00.012331 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45719
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:26:00.352979 18008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:00.353312 18008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:00.353468 18008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:00.360603 18008 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 04:26:00.360725 18008 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 04:26:00.360816 18008 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 04:26:00.360924 18008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:26:00.361025 18008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:00.361152 18008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:00.361241 18008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:26:00.361323 18008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:26:00.370903 18008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:00.372169 18008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:00.373471 18008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:00.385129 18013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:00.385071 18016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:00.385094 18014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:00.386116 18008 server_base.cc:1034] running on GCE node
I20260807 04:26:00.386919 18008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:00.388290 18008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:00.389549 18008 hybrid_clock.cc:648] HybridClock initialized: now 1786076760389496 us; error 54 us; skew 500 ppm
I20260807 04:26:00.389876 18008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:00 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076760, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:00.393031 18008 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:26:00.394424 18008 webserver.cc:460] Webserver started at http://127.1.244.254:38017/ using document root <none> and password file <none>
I20260807 04:26:00.395097 18008 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:00.395231 18008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:00.395558 18008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:26:00.398561 18008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "79ab166d1064403eb9c3a3139be37e9b"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "b0aa98aa497fb2ceeab7b73b41a6c540"
server_key_iv: "ce344ba191f28fde3fa8a2b3c62e934e"
server_key_version: "encryptionkey@0"
I20260807 04:26:00.399296 18008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "79ab166d1064403eb9c3a3139be37e9b"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "b0aa98aa497fb2ceeab7b73b41a6c540"
server_key_iv: "ce344ba191f28fde3fa8a2b3c62e934e"
server_key_version: "encryptionkey@0"
I20260807 04:26:00.404414 18008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 04:26:00.407801 18024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:00.409066 18008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:26:00.409235 18008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "79ab166d1064403eb9c3a3139be37e9b"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "b0aa98aa497fb2ceeab7b73b41a6c540"
server_key_iv: "ce344ba191f28fde3fa8a2b3c62e934e"
server_key_version: "encryptionkey@0"
I20260807 04:26:00.409447 18008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:00.427858 18008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:00.431754 18008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:00.432050 18008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:00.450927 18008 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45719
I20260807 04:26:00.450953 18077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45719 every 8 connection(s)
I20260807 04:26:00.452327 18008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:26:00.456167 18078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:00.457027 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18008
I20260807 04:26:00.457265 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:26:00.458180 2003 external_mini_cluster.cc:1428] Setting key 9a80b280635598e4c09d9d116b8cef6a
I20260807 04:26:00.470572 18078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap starting.
Aug 07 04:26:00 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:00.474344 18078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:00.475581 18078 log.cc:824] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:00.478466 18078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: No bootstrap required, opened a new log
I20260807 04:26:00.481935 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:00.460276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.1.244.254:45719)
0807 04:26:00.461133 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:00.461144 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:00.461192 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:26:00.462829 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:00.462850 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:00.463228 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:00.463554 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:00.465470 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:00.466746 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:00.468744 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:00.469402 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:00.469534 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:00.474499 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:00.474512 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:00.474534 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:00.474578 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:00.477832 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:00.478526 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:00.478532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:00.478538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:00.478766 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:00.479295 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:00.479300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:00.479303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:00.479818 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:00.480117 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:00.480556 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 04:26:00.480937 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":386,"threads_started":1}
I20260807 04:26:00.487360 18078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:00.487722 18078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:00.487828 18078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ab166d1064403eb9c3a3139be37e9b, State: Initialized, Role: FOLLOWER
I20260807 04:26:00.488390 18078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:00.488643 18078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:26:00.488796 18078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:26:00.488967 18078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:00.491058 18078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:00.491575 18078 leader_election.cc:302] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ab166d1064403eb9c3a3139be37e9b; no voters:
I20260807 04:26:00.492309 18078 leader_election.cc:288] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:26:00.492411 18083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:26:00.493093 18083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 LEADER]: Becoming Leader. State: Replica: 79ab166d1064403eb9c3a3139be37e9b, State: Running, Role: LEADER
I20260807 04:26:00.493664 18083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:00.494333 18078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:26:00.496649 18084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ab166d1064403eb9c3a3139be37e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } } }
I20260807 04:26:00.496824 18085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ab166d1064403eb9c3a3139be37e9b. Latest consensus state: current_term: 1 leader_uuid: "79ab166d1064403eb9c3a3139be37e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } } }
I20260807 04:26:00.497112 18084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: This master's current role is: LEADER
I20260807 04:26:00.497150 18085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: This master's current role is: LEADER
I20260807 04:26:00.499208 18092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:26:00.503648 18092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:26:00.512064 18092 catalog_manager.cc:1347] Generated new cluster ID: b1d2d01204cd4c668efddddb87b0ae92
I20260807 04:26:00.512178 18092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:26:00.529105 18092 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:26:00.530370 18092 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:26:00.537053 18092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Generated new TSK 0
I20260807 04:26:00.537992 18092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:26:00.609582 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45719
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:26:00.943643 18104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:00.943972 18104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:00.944144 18104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:00.951810 18104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:00.951970 18104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:00.952159 18104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:26:00.962803 18104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:00.964131 18104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:00.965497 18104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:00.978603 18110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:00.978760 18112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:00.978768 18109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:00.979688 18104 server_base.cc:1034] running on GCE node
I20260807 04:26:00.980466 18104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:00.981400 18104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:00.982621 18104 hybrid_clock.cc:648] HybridClock initialized: now 1786076760982561 us; error 37 us; skew 500 ppm
I20260807 04:26:00.982964 18104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:00 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076760, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:00.986171 18104 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:26:00.987658 18104 webserver.cc:460] Webserver started at http://127.1.244.193:42895/ using document root <none> and password file <none>
I20260807 04:26:00.988375 18104 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:00.988512 18104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:00.988828 18104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:26:00.991590 18104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "9370db9f44a94d17bb0b82e52dc71923"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "c66fab49ee2dd621660b84a8eae3c28f"
server_key_iv: "a253fa996fa73e249a00f9778dd96a06"
server_key_version: "encryptionkey@0"
I20260807 04:26:00.992360 18104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "9370db9f44a94d17bb0b82e52dc71923"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "c66fab49ee2dd621660b84a8eae3c28f"
server_key_iv: "a253fa996fa73e249a00f9778dd96a06"
server_key_version: "encryptionkey@0"
I20260807 04:26:00.997629 18104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:26:01.000986 18119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:01.002375 18104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:26:01.002566 18104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "9370db9f44a94d17bb0b82e52dc71923"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "c66fab49ee2dd621660b84a8eae3c28f"
server_key_iv: "a253fa996fa73e249a00f9778dd96a06"
server_key_version: "encryptionkey@0"
I20260807 04:26:01.002767 18104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:01.015115 18104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:01.019173 18104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:01.019466 18104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:01.020470 18104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:01.021752 18104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:01.021858 18104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:01.022022 18104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:01.022099 18104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:01.055996 18104 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:32837
I20260807 04:26:01.056057 18233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:32837 every 8 connection(s)
I20260807 04:26:01.057279 18104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:26:01.057852 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18104
I20260807 04:26:01.058110 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:26:01.058966 2003 external_mini_cluster.cc:1428] Setting key ec458163c407fc0b4c21ae82c0c9e8a5
Aug 07 04:26:01 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076760, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:01.079203 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:01.060101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:46859 (local address 127.1.244.254:45719)
0807 04:26:01.060593 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:01.060599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:01.061727 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 04:26:01.063573 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:01.063582 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:01.063668 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:01.063778 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:01.065474 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.066597 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.068802 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.069532 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.069678 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:01.073501 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:01.073506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:01.073509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:01.073541 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:01.076171 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:01.076951 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:01.076956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:01.076958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:01.077083 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:01.077532 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:01.077536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:01.077538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:01.077784 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:01.077948 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:01.078605 (+ 657us) server_negotiation.cc:299] Negotiation successful
0807 04:26:01.078825 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":253,"threads_started":1}
I20260807 04:26:01.080190 18237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:01.060711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:46859)
0807 04:26:01.061552 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:01.061599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:01.063281 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:01.064049 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:01.064058 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:01.064568 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:01.065285 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:01.065308 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.066742 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.066749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:01.068649 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:01.068666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.069749 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.069757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:01.070042 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:01.070795 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:01.070824 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:01.073276 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:01.076309 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:01.076317 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:01.076332 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:01.076803 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:01.077196 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:01.077202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:01.077205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:01.077395 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:01.077915 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:01.077923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:01.078242 (+ 319us) client_negotiation.cc:769] Sending connection context
0807 04:26:01.078495 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 04:26:01.078991 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":367,"threads_started":1}
I20260807 04:26:01.081765 18234 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45719
I20260807 04:26:01.082098 18234 heartbeater.cc:464] Registering TS with master...
I20260807 04:26:01.083002 18234 heartbeater.cc:511] Master 127.1.244.254:45719 requested a full tablet report, sending...
I20260807 04:26:01.085155 18041 ts_manager.cc:194] Registered new tserver with Master: 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193:32837)
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
I20260807 04:26:01.086591 18041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:46859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:26:01.119454 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45719
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:26:01.448478 18240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:01.448776 18240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:01.448969 18240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:01.456246 18240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:01.456344 18240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:01.456544 18240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:26:01.469270 18240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:01.470517 18240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:01.471819 18240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:01.485217 18245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:01.485278 18248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:01.485311 18246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:01.486049 18240 server_base.cc:1034] running on GCE node
I20260807 04:26:01.486583 18240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:01.487440 18240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:01.488669 18240 hybrid_clock.cc:648] HybridClock initialized: now 1786076761488626 us; error 52 us; skew 500 ppm
I20260807 04:26:01.488988 18240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:01 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:01.492067 18240 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:26:01.493378 18240 webserver.cc:460] Webserver started at http://127.1.244.194:46669/ using document root <none> and password file <none>
I20260807 04:26:01.494014 18240 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:01.494119 18240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:01.494369 18240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:26:01.497499 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "05c863845e2f4937b0ca26c92f7e84af"
format_stamp: "Formatted at 2026-08-07 04:26:01 on dist-test-slave-zkpd"
server_key: "25ad531e5b35df05bc0b345381604325"
server_key_iv: "c264b1c4e37847dc57d5fa36b4e76dc5"
server_key_version: "encryptionkey@0"
I20260807 04:26:01.498284 18240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "05c863845e2f4937b0ca26c92f7e84af"
format_stamp: "Formatted at 2026-08-07 04:26:01 on dist-test-slave-zkpd"
server_key: "25ad531e5b35df05bc0b345381604325"
server_key_iv: "c264b1c4e37847dc57d5fa36b4e76dc5"
server_key_version: "encryptionkey@0"
I20260807 04:26:01.503381 18240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:26:01.506899 18255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:01.508306 18240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:26:01.508482 18240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "05c863845e2f4937b0ca26c92f7e84af"
format_stamp: "Formatted at 2026-08-07 04:26:01 on dist-test-slave-zkpd"
server_key: "25ad531e5b35df05bc0b345381604325"
server_key_iv: "c264b1c4e37847dc57d5fa36b4e76dc5"
server_key_version: "encryptionkey@0"
I20260807 04:26:01.508667 18240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:01.530745 18240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:01.535092 18240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:01.535429 18240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:01.536442 18240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:01.537813 18240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:01.537928 18240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:01.538071 18240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:01.538144 18240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:01.570194 18240 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41865
I20260807 04:26:01.570276 18371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41865 every 8 connection(s)
I20260807 04:26:01.571491 18240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:26:01.576236 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18240
I20260807 04:26:01.576484 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:26:01.576946 2003 external_mini_cluster.cc:1428] Setting key 0f877934711ff52f96211e79ab4a690f
Aug 07 04:26:01 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076761, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:01.593098 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:01.574125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:48335 (local address 127.1.244.254:45719)
0807 04:26:01.574352 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:01.574358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:01.575460 (+ 1102us) server_negotiation.cc:407] Connection header received
0807 04:26:01.577525 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:01.577531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:01.577639 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:01.577753 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:01.579246 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.580062 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.582499 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.582902 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.582999 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:01.587144 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:01.587150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:01.587153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:01.587186 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:01.589814 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:01.590762 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:01.590767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:01.590769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:01.590896 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:01.591413 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:01.591416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:01.591418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:01.591670 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:01.591790 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:01.592686 (+ 896us) server_negotiation.cc:299] Negotiation successful
0807 04:26:01.592852 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:26:01.594805 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:01.574586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:48335)
0807 04:26:01.575287 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:01.575329 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:01.577252 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:01.577991 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:01.578000 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:01.578504 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:01.579062 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:01.579083 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.580266 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.580274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:01.582356 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:01.582373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:01.583115 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:01.583124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:01.583490 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:01.584253 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:01.584287 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:01.586914 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:01.589951 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:01.589960 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:01.589974 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:01.590622 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:01.591016 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:01.591022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:01.591025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:01.591296 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:01.591813 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:01.591823 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:01.592294 (+ 471us) client_negotiation.cc:769] Sending connection context
0807 04:26:01.592591 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 04:26:01.593160 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":270,"threads_started":1}
I20260807 04:26:01.596283 18372 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45719
I20260807 04:26:01.596585 18372 heartbeater.cc:464] Registering TS with master...
I20260807 04:26:01.597258 18372 heartbeater.cc:511] Master 127.1.244.254:45719 requested a full tablet report, sending...
I20260807 04:26:01.598760 18041 ts_manager.cc:194] Registered new tserver with Master: 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
I20260807 04:26:01.599550 18041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:48335
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:26:01.637259 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45719
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:26:01.964949 18377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:01.965260 18377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:01.965432 18377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:01.972611 18377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:01.972756 18377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:01.972929 18377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:26:01.982637 18377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:01.983930 18377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:01.985222 18377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:01.998009 18382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:01.998040 18383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:01.997927 18385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:01.998780 18377 server_base.cc:1034] running on GCE node
I20260807 04:26:01.999384 18377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:02.000344 18377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:02.001543 18377 hybrid_clock.cc:648] HybridClock initialized: now 1786076762001488 us; error 51 us; skew 500 ppm
I20260807 04:26:02.001853 18377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:02 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076762, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:02.005187 18377 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:26:02.006790 18377 webserver.cc:460] Webserver started at http://127.1.244.195:45855/ using document root <none> and password file <none>
I20260807 04:26:02.007448 18377 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:02.007596 18377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:02.007951 18377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:26:02.010861 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
format_stamp: "Formatted at 2026-08-07 04:26:02 on dist-test-slave-zkpd"
server_key: "9fb216e7dd558abd70a459555e4c48e4"
server_key_iv: "d3a39fc4d2103f7dbdd72ef4508139a7"
server_key_version: "encryptionkey@0"
I20260807 04:26:02.011605 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
format_stamp: "Formatted at 2026-08-07 04:26:02 on dist-test-slave-zkpd"
server_key: "9fb216e7dd558abd70a459555e4c48e4"
server_key_iv: "d3a39fc4d2103f7dbdd72ef4508139a7"
server_key_version: "encryptionkey@0"
I20260807 04:26:02.016987 18377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:26:02.020402 18392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:02.021632 18377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:26:02.021807 18377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
format_stamp: "Formatted at 2026-08-07 04:26:02 on dist-test-slave-zkpd"
server_key: "9fb216e7dd558abd70a459555e4c48e4"
server_key_iv: "d3a39fc4d2103f7dbdd72ef4508139a7"
server_key_version: "encryptionkey@0"
I20260807 04:26:02.022019 18377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:02.038136 18377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:02.041975 18377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:02.042255 18377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:02.043179 18377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:02.044459 18377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:02.044624 18377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:02.044780 18377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:02.044852 18377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:02.078291 18377 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41425
I20260807 04:26:02.078332 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41425 every 8 connection(s)
I20260807 04:26:02.079545 18377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:26:02.080087 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18377
I20260807 04:26:02.080317 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:26:02.080922 2003 external_mini_cluster.cc:1428] Setting key b5983ccdf77fa0975a8e737f746662ce
I20260807 04:26:02.089469 18234 heartbeater.cc:503] Master 127.1.244.254:45719 was elected leader, sending a full tablet report...
Aug 07 04:26:02 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076762, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:02.100135 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.082386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:55547 (local address 127.1.244.254:45719)
0807 04:26:02.082874 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.082879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.083984 (+ 1105us) server_negotiation.cc:407] Connection header received
0807 04:26:02.085825 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.085829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.085896 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.085973 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:02.087305 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.088144 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.089675 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.090266 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.090399 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.093788 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:02.093798 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:02.093801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:02.093833 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:02.097101 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.097765 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.097769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.097771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.097920 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.098361 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.098367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.098371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.098651 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:02.098813 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.099530 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.099743 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374}
I20260807 04:26:02.100973 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.082976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:55547)
0807 04:26:02.083800 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.083847 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:02.085557 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:02.086172 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:02.086179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:02.086642 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:02.087168 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.087185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.088284 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.088287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.089503 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.089515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.090471 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.090476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.090657 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.091400 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:02.091420 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:02.093578 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:02.097219 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:02.097225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:02.097236 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:02.097642 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:02.098059 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:02.098064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:02.098067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:02.098246 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:02.098804 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:02.098812 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:02.099164 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 04:26:02.099441 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 04:26:02.099948 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":339,"threads_started":1}
I20260807 04:26:02.102155 18510 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45719
I20260807 04:26:02.102465 18510 heartbeater.cc:464] Registering TS with master...
I20260807 04:26:02.103176 18510 heartbeater.cc:511] Master 127.1.244.254:45719 requested a full tablet report, sending...
I20260807 04:26:02.104660 18041 ts_manager.cc:194] Registered new tserver with Master: f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425)
I20260807 04:26:02.105451 18041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:55547
I20260807 04:26:02.108845 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:26:02.125283 18236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.113436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.1.244.254:45719)
0807 04:26:02.113584 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.113588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.114056 (+ 468us) server_negotiation.cc:407] Connection header received
0807 04:26:02.114175 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.114178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.114231 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.114320 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:02.115626 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.116411 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.117723 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.118125 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.118218 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.120668 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:02.120672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:02.120675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:02.120702 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:02.122968 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.123548 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.123552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.123554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.123673 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.124010 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.124013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.124015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.124306 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:02.124478 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.124778 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.124980 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:26:02.129161 18041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:26:02.135972 18041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:26:02.170127 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.148351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.1.244.193:32837)
0807 04:26:02.159279 (+ 10928us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.159293 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.161652 (+ 2359us) server_negotiation.cc:407] Connection header received
0807 04:26:02.162080 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.162091 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.162375 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.162570 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:02.163435 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.165172 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.167521 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.168922 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.169077 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.169328 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.169583 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.169752 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10669,"thread_start_us":10949,"threads_started":1}
I20260807 04:26:02.170259 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.148093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40414)
0807 04:26:02.161536 (+ 13443us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.161555 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:02.161865 (+ 310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:02.162791 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:02.162800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:02.162834 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:02.163217 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.163237 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.165313 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.165318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.167360 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.167372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.169122 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.169128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.169321 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.169381 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 04:26:02.169546 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 04:26:02.169732 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13309,"mutex_wait_us":135,"thread_start_us":12159,"threads_started":1}
I20260807 04:26:02.171205 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.147341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41425 (local address 127.0.0.1:41140)
0807 04:26:02.160638 (+ 13297us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.160686 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:02.160888 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:02.163462 (+ 2574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:02.163469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:02.163504 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:02.164032 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.164045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.166004 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.166010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.168229 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.168242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.170217 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.170223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.170524 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.170621 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 04:26:02.170696 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 04:26:02.170838 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13046,"mutex_wait_us":114,"thread_start_us":13933,"threads_started":1}
I20260807 04:26:02.171239 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.147777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.1.244.194:41865)
0807 04:26:02.160048 (+ 12271us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.160061 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.162520 (+ 2459us) server_negotiation.cc:407] Connection header received
0807 04:26:02.162830 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.162838 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.162998 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.163119 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:02.164130 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.165525 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.168209 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.169647 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.169787 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.170010 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.170794 (+ 784us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.170955 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12052,"thread_start_us":11950,"threads_started":1}
I20260807 04:26:02.172009 18525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.148359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41865 (local address 127.0.0.1:53098)
0807 04:26:02.162407 (+ 14048us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.162427 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:02.162624 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:02.163328 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:02.163332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:02.163380 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:02.163846 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.163907 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.165669 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.165673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.168018 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.168034 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.170191 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.170203 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.170521 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.170615 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 04:26:02.171474 (+ 859us) client_negotiation.cc:240] Negotiation successful
0807 04:26:02.171611 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13914,"mutex_wait_us":116,"thread_start_us":13760,"threads_started":1}
I20260807 04:26:02.172919 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.147644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.1.244.195:41425)
0807 04:26:02.162815 (+ 15171us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.162822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.162851 (+ 29us) server_negotiation.cc:407] Connection header received
0807 04:26:02.162959 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.162968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.163218 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.163407 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:02.164239 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.165855 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.168401 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.169749 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.169885 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.170080 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.172450 (+ 2370us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.172615 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15011,"thread_start_us":14833,"threads_started":1}
I20260807 04:26:02.174530 18168 tablet_service.cc:1467] Processing CreateTablet for tablet ddc59ab8efe84796bd43689651b7b90e (DEFAULT_TABLE table=test-table [id=76579da1e09745c08bee33cab3e186e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:26:02.175868 18168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc59ab8efe84796bd43689651b7b90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:02.175544 18304 tablet_service.cc:1467] Processing CreateTablet for tablet ddc59ab8efe84796bd43689651b7b90e (DEFAULT_TABLE table=test-table [id=76579da1e09745c08bee33cab3e186e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:26:02.176458 18443 tablet_service.cc:1467] Processing CreateTablet for tablet ddc59ab8efe84796bd43689651b7b90e (DEFAULT_TABLE table=test-table [id=76579da1e09745c08bee33cab3e186e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:26:02.177268 18304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc59ab8efe84796bd43689651b7b90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:02.177826 18443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc59ab8efe84796bd43689651b7b90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:02.193077 18529 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap starting.
I20260807 04:26:02.198088 18530 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Bootstrap starting.
I20260807 04:26:02.198568 18529 tablet_bootstrap.cc:654] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:02.199769 18531 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Bootstrap starting.
I20260807 04:26:02.199949 18529 log.cc:824] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:02.202630 18530 tablet_bootstrap.cc:654] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:02.203612 18530 log.cc:824] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:02.203938 18531 tablet_bootstrap.cc:654] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:02.205272 18531 log.cc:824] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:02.206679 18529 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: No bootstrap required, opened a new log
I20260807 04:26:02.207031 18529 ts_tablet_manager.cc:1397] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260807 04:26:02.207455 18530 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: No bootstrap required, opened a new log
I20260807 04:26:02.207701 18530 ts_tablet_manager.cc:1397] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 04:26:02.208739 18531 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: No bootstrap required, opened a new log
I20260807 04:26:02.208982 18531 ts_tablet_manager.cc:1397] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:26:02.216079 18530 raft_consensus.cc:348] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.216449 18530 raft_consensus.cc:374] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:02.216571 18530 raft_consensus.cc:729] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c863845e2f4937b0ca26c92f7e84af, State: Initialized, Role: FOLLOWER
I20260807 04:26:02.217173 18530 consensus_queue.cc:260] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.217742 18529 raft_consensus.cc:348] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.218219 18529 raft_consensus.cc:374] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:02.218271 18372 heartbeater.cc:503] Master 127.1.244.254:45719 was elected leader, sending a full tablet report...
I20260807 04:26:02.218400 18529 raft_consensus.cc:729] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9370db9f44a94d17bb0b82e52dc71923, State: Initialized, Role: FOLLOWER
I20260807 04:26:02.218963 18530 ts_tablet_manager.cc:1428] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260807 04:26:02.219136 18529 consensus_queue.cc:260] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.220927 18529 ts_tablet_manager.cc:1428] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260807 04:26:02.219846 18531 raft_consensus.cc:348] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.221546 18531 raft_consensus.cc:374] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:02.221673 18531 raft_consensus.cc:729] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76fbebe5a8e4ac78b53ff400359ad07, State: Initialized, Role: FOLLOWER
I20260807 04:26:02.222227 18531 consensus_queue.cc:260] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.223250 18510 heartbeater.cc:503] Master 127.1.244.254:45719 was elected leader, sending a full tablet report...
I20260807 04:26:02.223883 18531 ts_tablet_manager.cc:1428] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260807 04:26:02.288295 18536 raft_consensus.cc:482] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:02.288574 18536 raft_consensus.cc:504] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.290230 18536 leader_election.cc:288] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
I20260807 04:26:02.297039 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.290966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41425 (local address 127.1.244.193:40673)
0807 04:26:02.291630 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.291646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:02.291827 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:02.292196 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:02.292200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:02.292229 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:02.292551 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.292560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.293544 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.293551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.295116 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.295128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.296479 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.296483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.296600 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.296624 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:26:02.296702 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 04:26:02.296788 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":363,"threads_started":1}
I20260807 04:26:02.297150 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.290669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:40673 (local address 127.1.244.195:41425)
0807 04:26:02.290914 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.290921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.291738 (+ 817us) server_negotiation.cc:407] Connection header received
0807 04:26:02.292013 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.292017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.292077 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.292174 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:02.292688 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.293416 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.295286 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.296289 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.296447 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.296522 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.296752 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.296883 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:26:02.298316 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.290902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51545 (local address 127.1.244.194:41865)
0807 04:26:02.291076 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.291080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.292232 (+ 1152us) server_negotiation.cc:407] Connection header received
0807 04:26:02.292452 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.292457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.292516 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.292613 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:02.293142 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.293830 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.295824 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.297006 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.297165 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.297243 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.297946 (+ 703us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.298076 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:26:02.298422 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.290970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41865 (local address 127.1.244.193:51545)
0807 04:26:02.292124 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.292139 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:02.292292 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:02.292628 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:02.292632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:02.292656 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:02.292979 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.292988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.293983 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.293989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.295644 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:02.295657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.297239 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.297249 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:02.297745 (+ 496us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.297789 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 04:26:02.297893 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 04:26:02.298033 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":26,"thread_start_us":774,"threads_started":1}
I20260807 04:26:02.298573 18463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc59ab8efe84796bd43689651b7b90e" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" is_pre_election: true
I20260807 04:26:02.299206 18463 raft_consensus.cc:2454] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 0.
I20260807 04:26:02.299551 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc59ab8efe84796bd43689651b7b90e" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c863845e2f4937b0ca26c92f7e84af" is_pre_election: true
I20260807 04:26:02.300048 18122 leader_election.cc:302] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923, f76fbebe5a8e4ac78b53ff400359ad07; no voters:
I20260807 04:26:02.300099 18325 raft_consensus.cc:2454] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 0.
I20260807 04:26:02.300469 18536 raft_consensus.cc:2781] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:26:02.300642 18536 raft_consensus.cc:482] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:26:02.300757 18536 raft_consensus.cc:3037] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:02.302996 18536 raft_consensus.cc:504] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:02.303586 18536 leader_election.cc:288] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 election: Requested vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
I20260807 04:26:02.304036 18463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc59ab8efe84796bd43689651b7b90e" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
I20260807 04:26:02.304097 18325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc59ab8efe84796bd43689651b7b90e" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c863845e2f4937b0ca26c92f7e84af"
I20260807 04:26:02.304234 18463 raft_consensus.cc:3037] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:02.304350 18325 raft_consensus.cc:3037] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:02.306668 18325 raft_consensus.cc:2454] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 1.
I20260807 04:26:02.306725 18463 raft_consensus.cc:2454] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 1.
I20260807 04:26:02.307080 18121 leader_election.cc:302] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c863845e2f4937b0ca26c92f7e84af, 9370db9f44a94d17bb0b82e52dc71923; no voters:
I20260807 04:26:02.307411 18536 raft_consensus.cc:2781] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:26:02.307852 18536 raft_consensus.cc:686] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 LEADER]: Becoming Leader. State: Replica: 9370db9f44a94d17bb0b82e52dc71923, State: Running, Role: LEADER
I20260807 04:26:02.308409 18536 consensus_queue.cc:237] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
W20260807 04:26:02.309679 18235 tablet.cc:2369] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:26:02.313417 18039 catalog_manager.cc:5495] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "9370db9f44a94d17bb0b82e52dc71923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } health_report { overall_health: HEALTHY } } }
W20260807 04:26:02.323462 18373 tablet.cc:2369] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:26:02.331310 18511 tablet.cc:2369] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:26:02.340067 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.335022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.1.244.193:32837)
0807 04:26:02.335261 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.335265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.335284 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:26:02.335467 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.335472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.335565 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.335684 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:26:02.336038 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.336751 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.338104 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.338492 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.338582 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.339010 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:26:02.339481 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:26:02.339589 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.339693 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.339756 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 04:26:02.348027 18463 raft_consensus.cc:1261] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Refusing update from remote peer 9370db9f44a94d17bb0b82e52dc71923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:26:02.348037 18325 raft_consensus.cc:1261] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Refusing update from remote peer 9370db9f44a94d17bb0b82e52dc71923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:26:02.349020 18540 consensus_queue.cc:1035] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:26:02.349396 18536 consensus_queue.cc:1035] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:26:02.375432 18041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:26:02.375743 18041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:26:02.379458 18041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Sending DeleteTablet for 3 replicas of tablet ddc59ab8efe84796bd43689651b7b90e
I20260807 04:26:02.380542 18443 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc59ab8efe84796bd43689651b7b90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:02 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:41140
I20260807 04:26:02.380962 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18104
I20260807 04:26:02.380990 18168 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc59ab8efe84796bd43689651b7b90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:02 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:40414
I20260807 04:26:02.381268 18304 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc59ab8efe84796bd43689651b7b90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:02 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:53098
I20260807 04:26:02.390873 18555 tablet_replica.cc:330] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: stopping tablet replica
I20260807 04:26:02.391400 18555 raft_consensus.cc:2229] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:26:02.392011 18555 raft_consensus.cc:2258] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:02.396858 18553 tablet_replica.cc:330] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: stopping tablet replica
I20260807 04:26:02.397409 18553 raft_consensus.cc:2229] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:26:02.398102 18553 raft_consensus.cc:2258] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:02.399296 18555 ts_tablet_manager.cc:1905] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:26:02.400466 18553 ts_tablet_manager.cc:1905] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 04:26:02 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:26:02.406723 18554 tablet_replica.cc:330] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: stopping tablet replica
I20260807 04:26:02.407292 18554 raft_consensus.cc:2229] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:26:02.407970 18554 raft_consensus.cc:2258] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:02.408618 18553 ts_tablet_manager.cc:1918] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:26:02.408821 18553 log.cc:1196] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/ddc59ab8efe84796bd43689651b7b90e
I20260807 04:26:02.409415 18553 ts_tablet_manager.cc:1939] T ddc59ab8efe84796bd43689651b7b90e P f76fbebe5a8e4ac78b53ff400359ad07: Deleting consensus metadata
I20260807 04:26:02.410811 18555 ts_tablet_manager.cc:1918] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:26:02.410980 18555 log.cc:1196] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/ddc59ab8efe84796bd43689651b7b90e
I20260807 04:26:02.411528 18555 ts_tablet_manager.cc:1939] T ddc59ab8efe84796bd43689651b7b90e P 05c863845e2f4937b0ca26c92f7e84af: Deleting consensus metadata
I20260807 04:26:02.411617 18027 catalog_manager.cc:4841] TS f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): tablet ddc59ab8efe84796bd43689651b7b90e (table test-table [id=76579da1e09745c08bee33cab3e186e0]) successfully deleted
I20260807 04:26:02.413297 18026 catalog_manager.cc:4841] TS 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): tablet ddc59ab8efe84796bd43689651b7b90e (table test-table [id=76579da1e09745c08bee33cab3e186e0]) successfully deleted
I20260807 04:26:02.414031 18554 ts_tablet_manager.cc:1905] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:26:02.415964 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.382108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.1.244.193:32837)
0807 04:26:02.382275 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.382282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.394491 (+ 12209us) server_negotiation.cc:407] Connection header received
0807 04:26:02.394742 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.394748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.394852 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.394961 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:02.397246 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.398549 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.400684 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.401346 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.401455 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.407633 (+ 6178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:02.407649 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:02.407666 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:02.407711 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:02.412135 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.413173 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.413182 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.413185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.413370 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.413992 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.413997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.414001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.414506 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:02.414865 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.415223 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.415475 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:26:02.416627 18229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:02.530550 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18104
W20260807 04:26:02.551162 18028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:32837 (error 108)
I20260807 04:26:02.551290 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18240
I20260807 04:26:02.553184 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.552661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40446)
0807 04:26:02.552856 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.552999 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 04:26:02.553429 18028 catalog_manager.cc:4584] TS 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193:32837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ddc59ab8efe84796bd43689651b7b90e: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Aug 07 04:26:02 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:26:02.566133 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.552052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.1.244.194:41865)
0807 04:26:02.552226 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.552230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.552342 (+ 112us) server_negotiation.cc:407] Connection header received
0807 04:26:02.552562 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.552587 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.552666 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.552771 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:02.554274 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.555003 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.556401 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.556790 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.556881 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.561016 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:02.561029 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:02.561043 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:02.561079 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:02.563666 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.564354 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.564360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.564362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.564493 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.564817 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.564820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.564822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.565169 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:02.565379 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.565640 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.565796 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:26:02.566828 18367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:02.606997 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.606525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40462)
0807 04:26:02.606730 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.606822 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 04:26:02.608032 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.607615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40474)
0807 04:26:02.607762 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.607852 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 04:26:02.656100 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18240
I20260807 04:26:02.668718 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.668243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40482)
0807 04:26:02.668449 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.668558 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 04:26:02.669735 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.669367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40490)
0807 04:26:02.669518 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.669602 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 04:26:02.675506 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18377
Aug 07 04:26:02 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:26:02.690802 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.676273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.1.244.195:41425)
0807 04:26:02.676466 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:02.676473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:02.676496 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:26:02.676716 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:02.676720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:02.676786 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:02.676903 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:02.678164 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.678891 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.680267 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:02.680667 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:02.680807 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:02.685352 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:02.685367 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:02.685383 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:02.685416 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:02.688029 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.688745 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.688753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.688755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.688885 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:02.689266 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:02.689271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:02.689274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:02.689626 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:02.689949 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:02.690254 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 04:26:02.690478 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:26:02.691430 18505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:02.750415 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.749816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40498)
0807 04:26:02.750096 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.750215 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 04:26:02.751596 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:02.751123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:40504)
0807 04:26:02.751372 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:02.751463 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 04:26:02.780799 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18377
I20260807 04:26:02.799588 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18008
I20260807 04:26:02.800812 18073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:02.910493 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18008
I20260807 04:26:22.935499 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45719
--webserver_interface=127.1.244.254
--webserver_port=38017
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:26:23.276306 18563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:23.276597 18563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:23.276803 18563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:23.283839 18563 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 04:26:23.283977 18563 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 04:26:23.284049 18563 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 04:26:23.284145 18563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:26:23.284243 18563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:23.284371 18563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:23.284421 18563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:26:23.284520 18563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:26:23.294278 18563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=38017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:23.295527 18563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:23.296878 18563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:23.322511 18571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:23.322512 18572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:23.322535 18574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:23.323191 18563 server_base.cc:1034] running on GCE node
I20260807 04:26:23.324393 18563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:23.325997 18563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:23.327227 18563 hybrid_clock.cc:648] HybridClock initialized: now 1786076783327165 us; error 26 us; skew 500 ppm
I20260807 04:26:23.327627 18563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:23 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:23.336195 18563 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:26:23.337955 18563 webserver.cc:460] Webserver started at http://127.1.244.254:38017/ using document root <none> and password file <none>
I20260807 04:26:23.338752 18563 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:23.338891 18563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:23.345103 18563 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:26:23.348111 18581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:23.349455 18563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:26:23.349642 18563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "79ab166d1064403eb9c3a3139be37e9b"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "b0aa98aa497fb2ceeab7b73b41a6c540"
server_key_iv: "ce344ba191f28fde3fa8a2b3c62e934e"
server_key_version: "encryptionkey@0"
I20260807 04:26:23.350706 18563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:23.373451 18563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:23.377856 18563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:23.378216 18563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:23.398001 18563 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45719
I20260807 04:26:23.398025 18634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45719 every 8 connection(s)
I20260807 04:26:23.399577 18563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:26:23.404163 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18563
I20260807 04:26:23.404996 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:32837
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:45719
--webserver_port=42895
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:26:23.406915 18635 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:26:23.410398 18635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap starting.
I20260807 04:26:23.430895 18635 log.cc:824] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:23.443154 18635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:26:23.443666 18635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap complete.
I20260807 04:26:23.452790 18635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:23.453626 18635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ab166d1064403eb9c3a3139be37e9b, State: Initialized, Role: FOLLOWER
I20260807 04:26:23.454316 18635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:23.454608 18635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:26:23.454748 18635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:26:23.454967 18635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 1 FOLLOWER]: Advancing to term 2
I20260807 04:26:23.459338 18635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:23.460080 18635 leader_election.cc:302] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ab166d1064403eb9c3a3139be37e9b; no voters:
I20260807 04:26:23.460944 18635 leader_election.cc:288] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 04:26:23.461169 18639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Leader election won for term 2
I20260807 04:26:23.466816 18635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:26:23.470642 18639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 LEADER]: Becoming Leader. State: Replica: 79ab166d1064403eb9c3a3139be37e9b, State: Running, Role: LEADER
I20260807 04:26:23.471226 18639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:23.474047 18640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "79ab166d1064403eb9c3a3139be37e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } } }
I20260807 04:26:23.474429 18640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: This master's current role is: LEADER
I20260807 04:26:23.474181 18641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ab166d1064403eb9c3a3139be37e9b. Latest consensus state: current_term: 2 leader_uuid: "79ab166d1064403eb9c3a3139be37e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } } }
I20260807 04:26:23.474884 18641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: This master's current role is: LEADER
I20260807 04:26:23.481958 18655 catalog_manager.cc:1259] Loaded cluster ID: b1d2d01204cd4c668efddddb87b0ae92
I20260807 04:26:23.482110 18655 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: loading cluster ID for follower catalog manager: success
I20260807 04:26:23.486053 18655 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: acquiring CA information for follower catalog manager: success
I20260807 04:26:23.489914 18655 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:26:23.491189 18652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:26:23.495857 18652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:26:23.497068 18652 catalog_manager.cc:1259] Loaded cluster ID: b1d2d01204cd4c668efddddb87b0ae92
I20260807 04:26:23.497165 18652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:26:23.498807 18652 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:26:23.500084 18652 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Loaded TSK: 0
I20260807 04:26:23.537583 18652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Generated new TSK 1
I20260807 04:26:23.537985 18652 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:26:23.760634 18637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:23.760968 18637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:23.761128 18637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:23.768723 18637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:23.768882 18637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:23.769060 18637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:26:23.779179 18637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:32837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=42895
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:23.780617 18637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:23.782032 18637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:23.796826 18664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:23.796888 18663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:23.797891 18666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:23.798321 18637 server_base.cc:1034] running on GCE node
I20260807 04:26:23.799036 18637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:23.800252 18637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:23.801504 18637 hybrid_clock.cc:648] HybridClock initialized: now 1786076783801439 us; error 53 us; skew 500 ppm
I20260807 04:26:23.801911 18637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:23 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:23.805830 18637 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:26:23.807386 18637 webserver.cc:460] Webserver started at http://127.1.244.193:42895/ using document root <none> and password file <none>
I20260807 04:26:23.808195 18637 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:23.808357 18637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:23.814116 18637 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:26:23.817219 18673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:23.818542 18637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:26:23.818720 18637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "9370db9f44a94d17bb0b82e52dc71923"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "c66fab49ee2dd621660b84a8eae3c28f"
server_key_iv: "a253fa996fa73e249a00f9778dd96a06"
server_key_version: "encryptionkey@0"
I20260807 04:26:23.819681 18637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:23.843990 18637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:23.848352 18637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:23.848631 18637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:23.849572 18637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:23.851347 18680 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 04:26:23.857009 18637 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 04:26:23.857151 18637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 04:26:23.857283 18637 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 04:26:23.860392 18637 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 04:26:23.860505 18637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260807 04:26:23.860852 18680 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap starting.
I20260807 04:26:23.869561 18680 log.cc:824] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:23.884480 18680 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:26:23.885123 18680 tablet_bootstrap.cc:492] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap complete.
I20260807 04:26:23.885557 18680 ts_tablet_manager.cc:1397] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Time spent bootstrapping tablet: real 0.025s user 0.018s sys 0.004s
I20260807 04:26:23.894814 18680 raft_consensus.cc:348] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:23.896524 18680 raft_consensus.cc:729] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9370db9f44a94d17bb0b82e52dc71923, State: Initialized, Role: FOLLOWER
I20260807 04:26:23.897169 18680 consensus_queue.cc:260] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:23.898687 18680 ts_tablet_manager.cc:1428] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 04:26:23.908883 18637 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:32837
I20260807 04:26:23.908960 18795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:32837 every 8 connection(s)
I20260807 04:26:23.910144 18637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:26:23.914446 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18637
I20260807 04:26:23.915174 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:41865
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:45719
--webserver_port=46669
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:26:23 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076783, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:23.934739 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:23.913154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:56291 (local address 127.1.244.254:45719)
0807 04:26:23.913830 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:23.913846 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:23.914137 (+ 291us) server_negotiation.cc:407] Connection header received
0807 04:26:23.915087 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:23.915120 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:23.915525 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:23.915835 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:23.917982 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.919308 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.921818 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.922493 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.922630 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:23.926910 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:23.926919 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:23.926933 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:23.926972 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:23.930091 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:23.931167 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:23.931173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:23.931178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:23.931335 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:23.931852 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:23.931856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:23.931857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:23.932330 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:23.932495 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:23.933503 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0807 04:26:23.933819 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":302,"threads_started":1}
I20260807 04:26:23.935364 18798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:23.912969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:56291)
0807 04:26:23.913931 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:23.913994 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:23.914550 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:23.916023 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:23.916035 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:23.916929 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:23.917771 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:23.917798 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.919484 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.919492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:23.921672 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:23.921691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.922690 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.922699 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:23.923014 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:23.923783 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:23.923832 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:23.926648 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:23.930256 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:23.930268 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:23.930286 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:23.931027 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:23.931461 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:23.931470 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:23.931477 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:23.931731 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:23.932482 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:23.932493 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:23.932963 (+ 470us) client_negotiation.cc:769] Sending connection context
0807 04:26:23.933318 (+ 355us) client_negotiation.cc:240] Negotiation successful
0807 04:26:23.933916 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":413,"threads_started":1}
I20260807 04:26:23.938437 18796 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45719
I20260807 04:26:23.938737 18796 heartbeater.cc:464] Registering TS with master...
I20260807 04:26:23.939435 18796 heartbeater.cc:511] Master 127.1.244.254:45719 requested a full tablet report, sending...
I20260807 04:26:23.943081 18598 ts_manager.cc:194] Registered new tserver with Master: 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193:32837)
I20260807 04:26:23.949653 18598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:56291
I20260807 04:26:23.962615 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:23.948268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41096)
0807 04:26:23.955061 (+ 6793us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:23.955097 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:23.955262 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:23.955981 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:23.955986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:23.956020 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:23.956305 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:23.956340 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.958533 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.958538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:23.959874 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:23.959886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.961972 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.961977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:23.962125 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:23.962158 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:26:23.962253 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 04:26:23.962368 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6570,"thread_start_us":6457,"threads_started":1}
I20260807 04:26:23.962919 18802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:23.948354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.1.244.193:32837)
0807 04:26:23.954713 (+ 6359us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:23.954722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:23.955316 (+ 594us) server_negotiation.cc:407] Connection header received
0807 04:26:23.955478 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:23.955487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:23.955720 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:23.955956 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:23.956520 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.958318 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.960093 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:23.961684 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:23.961864 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:23.962094 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:23.962351 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 04:26:23.962517 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6187,"thread_start_us":5991,"threads_started":1}
I20260807 04:26:23.964286 18726 tablet_service.cc:1514] Processing DeleteTablet for tablet ddc59ab8efe84796bd43689651b7b90e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:02 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:41096
I20260807 04:26:23.967200 18803 tablet_replica.cc:330] stopping tablet replica
I20260807 04:26:23.967576 18803 raft_consensus.cc:2229] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:26:23.968025 18803 raft_consensus.cc:2258] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:23.969214 18803 ts_tablet_manager.cc:1905] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:26:23.976506 18803 ts_tablet_manager.cc:1918] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:26:23.976687 18803 log.cc:1196] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/ddc59ab8efe84796bd43689651b7b90e
I20260807 04:26:23.977108 18803 ts_tablet_manager.cc:1939] T ddc59ab8efe84796bd43689651b7b90e P 9370db9f44a94d17bb0b82e52dc71923: Deleting consensus metadata
I20260807 04:26:23.979120 18585 catalog_manager.cc:4841] TS 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193:32837): tablet ddc59ab8efe84796bd43689651b7b90e (table test-table [id=76579da1e09745c08bee33cab3e186e0]) successfully deleted
W20260807 04:26:24.266510 18800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:24.266774 18800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:24.266974 18800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:24.274283 18800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:24.274415 18800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:24.274602 18800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:26:24.284029 18800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:41865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=46669
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:24.285293 18800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:24.286614 18800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:24.299608 18809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:24.299576 18808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:24.300315 18811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:24.300772 18800 server_base.cc:1034] running on GCE node
I20260807 04:26:24.301383 18800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:24.302294 18800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:24.303498 18800 hybrid_clock.cc:648] HybridClock initialized: now 1786076784303459 us; error 29 us; skew 500 ppm
I20260807 04:26:24.303813 18800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:24 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076784, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:24.306983 18800 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:26:24.308521 18800 webserver.cc:460] Webserver started at http://127.1.244.194:46669/ using document root <none> and password file <none>
I20260807 04:26:24.309183 18800 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:24.309350 18800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:24.314769 18800 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 04:26:24.317655 18818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:24.318959 18800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:26:24.319141 18800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "05c863845e2f4937b0ca26c92f7e84af"
format_stamp: "Formatted at 2026-08-07 04:26:01 on dist-test-slave-zkpd"
server_key: "25ad531e5b35df05bc0b345381604325"
server_key_iv: "c264b1c4e37847dc57d5fa36b4e76dc5"
server_key_version: "encryptionkey@0"
I20260807 04:26:24.320199 18800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:24.337610 18800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:24.342043 18800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:24.342324 18800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:24.343303 18800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:24.344712 18800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:24.344810 18800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:24.344960 18800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:24.345057 18800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:24.378466 18800 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41865
I20260807 04:26:24.378539 18934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41865 every 8 connection(s)
I20260807 04:26:24.379949 18800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:26:24.380667 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18800
I20260807 04:26:24.381553 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:41425
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:45719
--webserver_port=45855
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 04:26:24 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076784, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:24.405556 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.382700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44607 (local address 127.1.244.254:45719)
0807 04:26:24.382930 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.382937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.384278 (+ 1341us) server_negotiation.cc:407] Connection header received
0807 04:26:24.387143 (+ 2865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.387147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.387216 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.387305 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:24.389439 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.390273 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.392772 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.393209 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.393314 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.397484 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:24.397493 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:24.397500 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:24.397551 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:24.401660 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.402569 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.402577 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.402584 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.402743 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.403285 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.403293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.403299 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.403771 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:24.403951 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:24.404757 (+ 806us) server_negotiation.cc:299] Negotiation successful
0807 04:26:24.405066 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:26:24.406718 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.383236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:44607)
0807 04:26:24.384080 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:24.384134 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:24.386838 (+ 2704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:24.387579 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:24.387593 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:24.388418 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:24.389259 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.389288 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.390466 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.390476 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.392625 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.392644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.393454 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.393464 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.393830 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.394610 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:24.394633 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:24.397151 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:24.401817 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:24.401834 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:24.401849 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:24.402407 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:24.402873 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:24.402878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:24.402882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:24.403139 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:24.403936 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:24.403944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:24.404330 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 04:26:24.404610 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 04:26:24.405142 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":304,"threads_started":1}
I20260807 04:26:24.408385 18935 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45719
I20260807 04:26:24.408751 18935 heartbeater.cc:464] Registering TS with master...
I20260807 04:26:24.409739 18935 heartbeater.cc:511] Master 127.1.244.254:45719 requested a full tablet report, sending...
I20260807 04:26:24.411583 18598 ts_manager.cc:194] Registered new tserver with Master: 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
I20260807 04:26:24.412803 18598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:44607
W20260807 04:26:24.727519 18938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:24.727841 18938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:24.728042 18938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:24.735387 18938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:24.735532 18938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:24.735724 18938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:26:24.745604 18938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:41425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=45855
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:24.746894 18938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:24.748226 18938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:24.762233 18943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:24.762182 18946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:24.762276 18944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:24.762965 18938 server_base.cc:1034] running on GCE node
I20260807 04:26:24.763546 18938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:24.764582 18938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:24.765903 18938 hybrid_clock.cc:648] HybridClock initialized: now 1786076784765847 us; error 59 us; skew 500 ppm
I20260807 04:26:24.766233 18938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:24 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076784, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:24.769546 18938 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:26:24.770956 18938 webserver.cc:460] Webserver started at http://127.1.244.195:45855/ using document root <none> and password file <none>
I20260807 04:26:24.771639 18938 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:24.771785 18938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:24.777041 18938 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:26:24.779647 18953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:24.780927 18938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:26:24.781107 18938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
format_stamp: "Formatted at 2026-08-07 04:26:02 on dist-test-slave-zkpd"
server_key: "9fb216e7dd558abd70a459555e4c48e4"
server_key_iv: "d3a39fc4d2103f7dbdd72ef4508139a7"
server_key_version: "encryptionkey@0"
I20260807 04:26:24.782056 18938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:24.805274 18938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:24.809198 18938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:24.809478 18938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:24.810734 18938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:24.812227 18938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:24.812350 18938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:24.812481 18938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:24.812577 18938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:24.846638 18938 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41425
I20260807 04:26:24.846746 19069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41425 every 8 connection(s)
I20260807 04:26:24.848114 18938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:26:24.856065 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 18938
Aug 07 04:26:24 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076784, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:26:24.872141 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.850667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:56765 (local address 127.1.244.254:45719)
0807 04:26:24.850866 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.850871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.851941 (+ 1070us) server_negotiation.cc:407] Connection header received
0807 04:26:24.853851 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.853858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.853954 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.854078 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:24.855606 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.856421 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.858489 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.858907 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.859043 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.862877 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:24.862886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:24.862889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:24.862920 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:24.866596 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.867538 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.867545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.867550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.867714 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.868259 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.868263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.868265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.870374 (+ 2109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:24.870531 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:24.871495 (+ 964us) server_negotiation.cc:299] Negotiation successful
0807 04:26:24.871758 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:26:24.872859 19073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.856951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.1.244.254:45719)
0807 04:26:24.857396 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.857401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.857807 (+ 406us) server_negotiation.cc:407] Connection header received
0807 04:26:24.857940 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.857947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.858022 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.858132 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:24.859575 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.860761 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.862288 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.862946 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.863052 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.866782 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:24.866789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:24.866793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:24.866833 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:24.869786 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.870500 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.870520 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.870525 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.870676 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.871126 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.871129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.871131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.871459 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:24.871765 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:24.872109 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 04:26:24.872374 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":228,"threads_started":1}
I20260807 04:26:24.873162 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.851076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:56765)
0807 04:26:24.851757 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:24.851796 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:24.853549 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:24.854323 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:24.854330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:24.854807 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:24.855418 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.855437 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.856591 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.856596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.858350 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.858362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.859131 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.859138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.859380 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.860160 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:24.860188 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:24.862666 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:24.866741 (+ 4075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:24.866750 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:24.866764 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:24.867418 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:24.867842 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:24.867848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:24.867851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:24.868135 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:24.870607 (+ 2472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:24.870612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:24.870994 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 04:26:24.871330 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 04:26:24.871806 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":266,"threads_started":1}
I20260807 04:26:24.874588 19070 heartbeater.cc:347] Connected to a master server at 127.1.244.254:45719
I20260807 04:26:24.874902 19070 heartbeater.cc:464] Registering TS with master...
I20260807 04:26:24.875638 19070 heartbeater.cc:511] Master 127.1.244.254:45719 requested a full tablet report, sending...
I20260807 04:26:24.877048 18598 ts_manager.cc:194] Registered new tserver with Master: f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425)
I20260807 04:26:24.877775 18598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:56765
I20260807 04:26:24.887483 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:26:24.893710 19073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.888421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43522 (local address 127.1.244.254:45719)
0807 04:26:24.888623 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.888632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.889097 (+ 465us) server_negotiation.cc:407] Connection header received
0807 04:26:24.889247 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.889250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.889375 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.889490 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:26:24.889881 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.890516 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.891980 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.892386 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.892486 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.892947 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:26:24.893193 (+ 246us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 04:26:24.893447 (+ 254us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43522: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 04:26:24.893831 19073 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43522: token expired
W20260807 04:26:24.894354 18513 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.244.254:45719)
I20260807 04:26:24.905973 19073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.894851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.1.244.254:45719)
0807 04:26:24.895025 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.895029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.895043 (+ 14us) server_negotiation.cc:407] Connection header received
0807 04:26:24.895165 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.895168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.895217 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.895287 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:24.896684 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.897327 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.898603 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.899007 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.899093 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.901543 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:24.901548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:24.901551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:24.901579 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:24.903860 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.904470 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.904474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.904476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.904619 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:24.904922 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:24.904925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:24.904926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:24.905203 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:24.905386 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:24.905606 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 04:26:24.905753 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:26:24.909289 18598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:26:24.916777 18598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:26:24.928503 18726 tablet_service.cc:1467] Processing CreateTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c (DEFAULT_TABLE table=test-table [id=ed14a4e03dcd4bb78dec67254cae9ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:26:24.929483 18726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057d4f8954d14ae0baafc1feccbd0c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:24.940902 19086 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap starting.
I20260807 04:26:24.942585 19086 tablet_bootstrap.cc:654] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:24.944953 19086 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: No bootstrap required, opened a new log
I20260807 04:26:24.945113 19086 ts_tablet_manager.cc:1397] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 04:26:24.945945 19086 raft_consensus.cc:348] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.946210 19086 raft_consensus.cc:374] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:24.946333 19086 raft_consensus.cc:729] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9370db9f44a94d17bb0b82e52dc71923, State: Initialized, Role: FOLLOWER
I20260807 04:26:24.946569 19086 consensus_queue.cc:260] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.948019 19086 ts_tablet_manager.cc:1428] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 04:26:24.948413 18796 heartbeater.cc:503] Master 127.1.244.254:45719 was elected leader, sending a full tablet report...
I20260807 04:26:24.948472 19082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.926067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41425 (local address 127.0.0.1:35656)
0807 04:26:24.926796 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:24.926823 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:24.926990 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:24.941372 (+ 14382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:24.941378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:24.941414 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:24.941912 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.941921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.943815 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.943821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.945805 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.945834 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.947681 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.947690 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.947865 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.947933 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 04:26:24.948006 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 04:26:24.948135 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":132,"spinlock_wait_cycles":1408,"thread_start_us":308,"threads_started":1}
I20260807 04:26:24.949726 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.926320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.1.244.195:41425)
0807 04:26:24.940516 (+ 14196us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.940523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.940557 (+ 34us) server_negotiation.cc:407] Connection header received
0807 04:26:24.940661 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.940670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.940922 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.941109 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:24.942112 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.943621 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.945985 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.947431 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.948952 (+ 1521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.949160 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:24.949259 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 04:26:24.949417 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14018,"thread_start_us":13870,"threads_started":1}
I20260807 04:26:24.949833 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.926859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.1.244.194:41865)
0807 04:26:24.939581 (+ 12722us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.939593 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.940629 (+ 1036us) server_negotiation.cc:407] Connection header received
0807 04:26:24.940720 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.940726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.940918 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.941109 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:24.942064 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.944178 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.946938 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.948771 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.948926 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.949125 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:24.949357 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 04:26:24.949546 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12510,"thread_start_us":13104,"threads_started":1}
I20260807 04:26:24.949856 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.926734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41865 (local address 127.0.0.1:60934)
0807 04:26:24.940359 (+ 13625us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:24.940379 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:24.940505 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:24.941372 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:24.941378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:24.941421 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:24.941832 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.941852 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.944388 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.944397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.946549 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.946567 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.949009 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.949019 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:24.949181 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:24.949219 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:26:24.949309 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 04:26:24.949430 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13490,"mutex_wait_us":5,"spinlock_wait_cycles":8832,"thread_start_us":13081,"threads_started":1}
I20260807 04:26:24.953343 19002 tablet_service.cc:1467] Processing CreateTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c (DEFAULT_TABLE table=test-table [id=ed14a4e03dcd4bb78dec67254cae9ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:26:24.953333 18868 tablet_service.cc:1467] Processing CreateTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c (DEFAULT_TABLE table=test-table [id=ed14a4e03dcd4bb78dec67254cae9ac1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:26:24.954571 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057d4f8954d14ae0baafc1feccbd0c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:24.954602 18868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 057d4f8954d14ae0baafc1feccbd0c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:26:24.966120 19090 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Bootstrap starting.
I20260807 04:26:24.966467 19089 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Bootstrap starting.
I20260807 04:26:24.969281 19090 tablet_bootstrap.cc:654] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:24.969451 19089 tablet_bootstrap.cc:654] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Neither blocks nor log segments found. Creating new log.
I20260807 04:26:24.970292 19090 log.cc:824] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:24.970396 19089 log.cc:824] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:24.973155 19090 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: No bootstrap required, opened a new log
I20260807 04:26:24.973157 19089 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: No bootstrap required, opened a new log
I20260807 04:26:24.973417 19090 ts_tablet_manager.cc:1397] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 04:26:24.973453 19089 ts_tablet_manager.cc:1397] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260807 04:26:24.980394 19090 raft_consensus.cc:348] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.980376 19089 raft_consensus.cc:348] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.980702 19089 raft_consensus.cc:374] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:24.980719 19090 raft_consensus.cc:374] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:26:24.980829 19090 raft_consensus.cc:729] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76fbebe5a8e4ac78b53ff400359ad07, State: Initialized, Role: FOLLOWER
I20260807 04:26:24.980814 19089 raft_consensus.cc:729] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c863845e2f4937b0ca26c92f7e84af, State: Initialized, Role: FOLLOWER
I20260807 04:26:24.981338 19089 consensus_queue.cc:260] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.981417 19090 consensus_queue.cc:260] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.982390 18935 heartbeater.cc:503] Master 127.1.244.254:45719 was elected leader, sending a full tablet report...
I20260807 04:26:24.982878 19090 ts_tablet_manager.cc:1428] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20260807 04:26:24.983017 19089 ts_tablet_manager.cc:1428] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Time spent starting tablet: real 0.009s user 0.004s sys 0.006s
I20260807 04:26:24.983523 19070 heartbeater.cc:503] Master 127.1.244.254:45719 was elected leader, sending a full tablet report...
I20260807 04:26:24.992957 19088 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:24.993201 19088 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:24.994663 19088 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
I20260807 04:26:25.003366 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.995079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:58661 (local address 127.1.244.194:41865)
0807 04:26:24.995295 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.995304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.995948 (+ 644us) server_negotiation.cc:407] Connection header received
0807 04:26:24.996143 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.996147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.996200 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.996295 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:24.997010 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.998072 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.000958 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.002197 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.002352 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.002421 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:25.002989 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 04:26:25.003118 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:26:25.003376 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.995022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:45595 (local address 127.1.244.195:41425)
0807 04:26:24.995210 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:24.995218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:24.995811 (+ 593us) server_negotiation.cc:407] Connection header received
0807 04:26:24.996067 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:24.996074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:24.996132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:24.996223 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:26:24.996855 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.998060 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.000958 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.002197 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.002365 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.002429 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:25.002989 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 04:26:25.003118 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:26:25.003613 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.995094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41865 (local address 127.1.244.193:58661)
0807 04:26:24.995814 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:24.995829 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:24.995999 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:24.996311 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:24.996315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:24.996344 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:24.996815 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.996825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.998204 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.998210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:25.000792 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:25.000808 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.002392 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.002398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:25.002715 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.002835 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 04:26:25.002938 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 04:26:25.003100 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":271,"thread_start_us":431,"threads_started":1}
I20260807 04:26:25.003654 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:24.995028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41425 (local address 127.1.244.193:45595)
0807 04:26:24.995691 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:24.995715 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:24.995940 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:24.996243 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:24.996247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:24.996273 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:26:24.996688 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:24.996702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:24.998206 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:24.998210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:25.000792 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:25.000808 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.002397 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.002401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:25.002713 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.002839 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 04:26:25.002941 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 04:26:25.003099 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":293,"spinlock_wait_cycles":18176,"thread_start_us":336,"threads_started":1}
I20260807 04:26:25.004762 18888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057d4f8954d14ae0baafc1feccbd0c9c" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c863845e2f4937b0ca26c92f7e84af" is_pre_election: true
I20260807 04:26:25.004763 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057d4f8954d14ae0baafc1feccbd0c9c" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" is_pre_election: true
I20260807 04:26:25.005286 19022 raft_consensus.cc:2454] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 0.
I20260807 04:26:25.005286 18888 raft_consensus.cc:2454] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 0.
I20260807 04:26:25.005981 18675 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c863845e2f4937b0ca26c92f7e84af, 9370db9f44a94d17bb0b82e52dc71923; no voters:
I20260807 04:26:25.006424 19088 raft_consensus.cc:2781] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:26:25.006579 19088 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:26:25.006688 19088 raft_consensus.cc:3037] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:25.008982 19088 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:25.009552 19088 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 election: Requested vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
I20260807 04:26:25.009999 18888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057d4f8954d14ae0baafc1feccbd0c9c" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05c863845e2f4937b0ca26c92f7e84af"
I20260807 04:26:25.010039 19022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "057d4f8954d14ae0baafc1feccbd0c9c" candidate_uuid: "9370db9f44a94d17bb0b82e52dc71923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
I20260807 04:26:25.010214 18888 raft_consensus.cc:3037] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:25.010265 19022 raft_consensus.cc:3037] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:26:25.012576 18888 raft_consensus.cc:2454] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 1.
I20260807 04:26:25.012776 19022 raft_consensus.cc:2454] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9370db9f44a94d17bb0b82e52dc71923 in term 1.
I20260807 04:26:25.012995 18675 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05c863845e2f4937b0ca26c92f7e84af, 9370db9f44a94d17bb0b82e52dc71923; no voters:
I20260807 04:26:25.013294 19088 raft_consensus.cc:2781] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:26:25.013782 19088 raft_consensus.cc:686] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 LEADER]: Becoming Leader. State: Replica: 9370db9f44a94d17bb0b82e52dc71923, State: Running, Role: LEADER
I20260807 04:26:25.014196 19088 consensus_queue.cc:237] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:25.018287 18598 catalog_manager.cc:5495] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "9370db9f44a94d17bb0b82e52dc71923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } health_report { overall_health: HEALTHY } } }
I20260807 04:26:25.045938 19100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.038818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.1.244.193:32837)
0807 04:26:25.039365 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:25.039369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:25.039387 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:26:25.039466 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:25.039470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:25.039541 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:25.039668 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:26:25.040087 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.041035 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.043185 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.043876 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.044045 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.044551 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:26:25.045118 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:26:25.045296 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:25.045418 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 04:26:25.045501 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":306,"threads_started":1}
I20260807 04:26:25.051643 18888 raft_consensus.cc:1261] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Refusing update from remote peer 9370db9f44a94d17bb0b82e52dc71923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:26:25.051684 19022 raft_consensus.cc:1261] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Refusing update from remote peer 9370db9f44a94d17bb0b82e52dc71923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:26:25.052663 19098 consensus_queue.cc:1035] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:26:25.052977 19088 consensus_queue.cc:1035] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:26:25.065982 19103 mvcc.cc:205] Tried to move back new op lower bound from 7315770511563046912 to 7315770511421235200. Current Snapshot: MvccSnapshot[applied={T|T < 7315770511563046912}]
I20260807 04:26:25.079303 18598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:26:25.079588 18598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:26:25.084818 18598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Sending DeleteTablet for 3 replicas of tablet 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:25.086019 18868 tablet_service.cc:1514] Processing DeleteTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:60934
I20260807 04:26:25.086090 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:35656
I20260807 04:26:25.086230 18867 tablet_service.cc:1514] Processing DeleteTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:60934
I20260807 04:26:25.087179 18726 tablet_service.cc:1514] Processing DeleteTablet for tablet 057d4f8954d14ae0baafc1feccbd0c9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:26:25 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:41096
I20260807 04:26:25.087397 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18637
I20260807 04:26:25.111070 19112 tablet_replica.cc:330] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: stopping tablet replica
I20260807 04:26:25.111660 19112 raft_consensus.cc:2229] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:26:25.112327 19112 raft_consensus.cc:2258] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:25.113044 19100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.088694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.1.244.193:32837)
0807 04:26:25.088905 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:25.088912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:25.088934 (+ 22us) server_negotiation.cc:407] Connection header received
0807 04:26:25.088998 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:25.089006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:25.089102 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:25.089215 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:25.091693 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.093292 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.096550 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.097518 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.097667 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.102431 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:25.102449 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:25.102472 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:25.102546 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:25.107967 (+ 5421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:25.109087 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:25.109099 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:25.109106 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:25.109303 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:25.109882 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:25.109890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:25.109896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:25.110557 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:25.110920 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:25.112125 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0807 04:26:25.112465 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:26:25.114588 19112 ts_tablet_manager.cc:1905] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:26:25.114449 19113 tablet_replica.cc:330] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: stopping tablet replica
I20260807 04:26:25.115396 18790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:25.116076 19111 tablet_replica.cc:330] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: stopping tablet replica
I20260807 04:26:25.116622 19111 raft_consensus.cc:2229] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:26:25.117182 19111 raft_consensus.cc:2258] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:25.120566 19111 ts_tablet_manager.cc:1905] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:26:25.122598 19112 ts_tablet_manager.cc:1918] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:26:25.122762 19112 log.cc:1196] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:25.123345 19112 ts_tablet_manager.cc:1939] T 057d4f8954d14ae0baafc1feccbd0c9c P f76fbebe5a8e4ac78b53ff400359ad07: Deleting consensus metadata
I20260807 04:26:25.125279 18584 catalog_manager.cc:4841] TS f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): tablet 057d4f8954d14ae0baafc1feccbd0c9c (table test-table [id=ed14a4e03dcd4bb78dec67254cae9ac1]) successfully deleted
I20260807 04:26:25.126670 19111 ts_tablet_manager.cc:1918] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:26:25.126784 19111 log.cc:1196] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:25.127247 19111 ts_tablet_manager.cc:1939] T 057d4f8954d14ae0baafc1feccbd0c9c P 05c863845e2f4937b0ca26c92f7e84af: Deleting consensus metadata
I20260807 04:26:25.128748 18583 catalog_manager.cc:4841] TS 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): tablet 057d4f8954d14ae0baafc1feccbd0c9c (table test-table [id=ed14a4e03dcd4bb78dec67254cae9ac1]) successfully deleted
W20260807 04:26:25.129179 18583 catalog_manager.cc:4808] TS 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): delete failed for tablet 057d4f8954d14ae0baafc1feccbd0c9c because the tablet was not found. No further retry: Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:25.115460 19113 raft_consensus.cc:2229] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:26:25.215242 19113 raft_consensus.cc:2258] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:26:25.215369 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18637
W20260807 04:26:25.235833 18585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:32837 (error 108)
I20260807 04:26:25.236124 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18800
I20260807 04:26:25.237879 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.237485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41112)
0807 04:26:25.237646 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.237735 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 04:26:25.238161 18585 catalog_manager.cc:4584] TS 9370db9f44a94d17bb0b82e52dc71923 (127.1.244.193:32837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 057d4f8954d14ae0baafc1feccbd0c9c: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
I20260807 04:26:25.250196 19085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.236972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.1.244.194:41865)
0807 04:26:25.237167 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:25.237171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:25.237271 (+ 100us) server_negotiation.cc:407] Connection header received
0807 04:26:25.237482 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:25.237487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:25.237574 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:25.237684 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:25.239164 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.239970 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.241370 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.241795 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.241923 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.244625 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:25.244646 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:25.244660 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:25.244692 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:25.247363 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:25.248128 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:25.248143 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:25.248145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:25.248263 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:25.248658 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:25.248663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:25.248666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:25.249027 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:25.249363 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:25.249631 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 04:26:25.249854 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:26:25.250833 18929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:25.291942 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.291357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41122)
0807 04:26:25.291593 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.291714 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 04:26:25.292991 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.292613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41134)
0807 04:26:25.292764 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.292863 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 04:26:25.339710 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18800
I20260807 04:26:25.353626 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.352943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41144)
0807 04:26:25.353249 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.353389 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 04:26:25.354866 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.354429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41150)
0807 04:26:25.354606 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.354710 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 04:26:25.360357 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18938
I20260807 04:26:25.374181 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.361175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.1.244.195:41425)
0807 04:26:25.361369 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:25.361375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:25.361396 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:26:25.361466 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:25.361472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:25.361545 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:25.361648 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:25.362964 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.363714 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.365115 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:25.365556 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:25.365693 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:25.368564 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:25.368578 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:25.368592 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:25.368629 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:25.371276 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:25.371999 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:25.372006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:25.372008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:25.372130 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:25.372510 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:25.372514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:25.372516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:25.372972 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:25.373272 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:25.373586 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 04:26:25.373867 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:26:25.374804 19063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:25.435243 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.434744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41152)
0807 04:26:25.434988 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.435085 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 04:26:25.436400 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.435996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41162)
0807 04:26:25.436232 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.436303 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260807 04:26:25.480396 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18938
I20260807 04:26:25.498433 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 18563
I20260807 04:26:25.499576 18630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:26:25.616407 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:25.615842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:32837 (local address 127.0.0.1:41170)
0807 04:26:25.616062 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:25.616190 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:32837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 04:26:25.616392 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 18563
I20260807 04:26:45.641438 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45719
--webserver_interface=127.1.244.254
--webserver_port=38017
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:45719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:26:45.975274 19119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:45.975522 19119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:45.975677 19119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:45.982580 19119 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 04:26:45.982666 19119 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 04:26:45.982726 19119 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 04:26:45.982769 19119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:26:45.982867 19119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:45.982985 19119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:45.983049 19119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:26:45.983144 19119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:26:45.992466 19119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:45719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:45719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=38017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:45.993644 19119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:45.994935 19119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:46.006691 19127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:46.006732 19125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:46.006922 19124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:46.007380 19119 server_base.cc:1034] running on GCE node
I20260807 04:26:46.008212 19119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:46.009506 19119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:46.010748 19119 hybrid_clock.cc:648] HybridClock initialized: now 1786076806010683 us; error 39 us; skew 500 ppm
I20260807 04:26:46.011071 19119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:46 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:46.014281 19119 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:26:46.015672 19119 webserver.cc:460] Webserver started at http://127.1.244.254:38017/ using document root <none> and password file <none>
I20260807 04:26:46.016394 19119 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:46.016523 19119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:46.021638 19119 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:26:46.024360 19134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:46.025605 19119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:26:46.025789 19119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "79ab166d1064403eb9c3a3139be37e9b"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "b0aa98aa497fb2ceeab7b73b41a6c540"
server_key_iv: "ce344ba191f28fde3fa8a2b3c62e934e"
server_key_version: "encryptionkey@0"
I20260807 04:26:46.026695 19119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:46.044952 19119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:46.048775 19119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:46.049069 19119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:46.067245 19119 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:45719
I20260807 04:26:46.067286 19186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:45719 every 8 connection(s)
I20260807 04:26:46.068769 19119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:26:46.075485 19187 sys_catalog.cc:263] Verifying existing consensus state
I20260807 04:26:46.078284 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19119
I20260807 04:26:46.078730 19187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap starting.
I20260807 04:26:46.079159 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:32837
--local_ip_for_outbound_sockets=127.1.244.193
--tserver_master_addrs=127.1.244.254:45719
--webserver_port=42895
--webserver_interface=127.1.244.193
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:26:46.098995 19187 log.cc:824] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:46.115461 19187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:26:46.116074 19187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Bootstrap complete.
I20260807 04:26:46.126802 19187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:46.127698 19187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ab166d1064403eb9c3a3139be37e9b, State: Initialized, Role: FOLLOWER
I20260807 04:26:46.128446 19187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:46.128696 19187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:26:46.128816 19187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:26:46.129022 19187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 2 FOLLOWER]: Advancing to term 3
I20260807 04:26:46.133164 19187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:46.133652 19187 leader_election.cc:302] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ab166d1064403eb9c3a3139be37e9b; no voters:
I20260807 04:26:46.134351 19187 leader_election.cc:288] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 04:26:46.134500 19192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 3 FOLLOWER]: Leader election won for term 3
I20260807 04:26:46.136176 19192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [term 3 LEADER]: Becoming Leader. State: Replica: 79ab166d1064403eb9c3a3139be37e9b, State: Running, Role: LEADER
I20260807 04:26:46.136775 19187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:26:46.136826 19192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } }
I20260807 04:26:46.139309 19194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ab166d1064403eb9c3a3139be37e9b. Latest consensus state: current_term: 3 leader_uuid: "79ab166d1064403eb9c3a3139be37e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } } }
I20260807 04:26:46.139684 19194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: This master's current role is: LEADER
I20260807 04:26:46.140221 19193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "79ab166d1064403eb9c3a3139be37e9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ab166d1064403eb9c3a3139be37e9b" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 45719 } } }
I20260807 04:26:46.140518 19193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b [sys.catalog]: This master's current role is: LEADER
I20260807 04:26:46.142725 19200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:26:46.151117 19200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:26:46.152597 19200 catalog_manager.cc:1259] Loaded cluster ID: b1d2d01204cd4c668efddddb87b0ae92
I20260807 04:26:46.152741 19200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:26:46.156672 19200 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:26:46.158313 19200 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Loaded TSK: 0
I20260807 04:26:46.158531 19200 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79ab166d1064403eb9c3a3139be37e9b: Loaded TSK: 1
F20260807 04:26:46.159682 19200 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fac519e79ad google::LogMessage::Fail() at ??:0
@ 0x7fac519ea88a google::LogMessage::SendToLog() at ??:0
@ 0x7fac519e74de google::LogMessage::Flush() at ??:0
@ 0x7fac519e8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fac5574b7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fac5573a73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fac6741a1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fac67419783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fac674a52a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fac674a50c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fac674ef5e9 std::function<>::operator()() at ??:0
@ 0x7fac6741dcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fac6741bc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fac674a3610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fac674a3439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fac54633345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fac5463691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fac54636799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fac54611eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fac578eb6db start_thread at ??:0
@ 0x7fac5088161f clone at ??:0
W20260807 04:26:46.482252 19189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:46.482513 19189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:46.482616 19189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:46.489535 19189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:46.489634 19189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:46.489848 19189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:26:46.499022 19189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:32837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=42895
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:46.500276 19189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:46.501561 19189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:46.517655 19238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:46.517912 19240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:46.518236 19237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:46.518988 19189 server_base.cc:1034] running on GCE node
I20260807 04:26:46.519764 19189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:46.520871 19189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:46.522132 19189 hybrid_clock.cc:648] HybridClock initialized: now 1786076806522044 us; error 63 us; skew 500 ppm
I20260807 04:26:46.522528 19189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:46 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:46.526268 19189 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:26:46.528265 19189 webserver.cc:460] Webserver started at http://127.1.244.193:42895/ using document root <none> and password file <none>
I20260807 04:26:46.528941 19189 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:46.529075 19189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:46.534319 19189 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 04:26:46.537073 19247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:46.538317 19189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:26:46.538496 19189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "9370db9f44a94d17bb0b82e52dc71923"
format_stamp: "Formatted at 2026-08-07 04:26:00 on dist-test-slave-zkpd"
server_key: "c66fab49ee2dd621660b84a8eae3c28f"
server_key_iv: "a253fa996fa73e249a00f9778dd96a06"
server_key_version: "encryptionkey@0"
I20260807 04:26:46.539460 19189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:46.560421 19189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:46.564883 19189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:46.565168 19189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:46.566126 19189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:46.568315 19254 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 04:26:46.573803 19189 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 04:26:46.573928 19189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 04:26:46.574074 19189 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 04:26:46.577108 19189 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 04:26:46.577210 19189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260807 04:26:46.577549 19254 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap starting.
I20260807 04:26:46.605585 19254 log.cc:824] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Log is configured to *not* fsync() on all Append() calls
I20260807 04:26:46.617344 19189 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:32837
I20260807 04:26:46.617373 19365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:32837 every 8 connection(s)
I20260807 04:26:46.618736 19189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:26:46.619256 19254 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 04:26:46.620061 19254 tablet_bootstrap.cc:492] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Bootstrap complete.
I20260807 04:26:46.621920 19254 ts_tablet_manager.cc:1397] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Time spent bootstrapping tablet: real 0.045s user 0.018s sys 0.024s
I20260807 04:26:46.623817 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:46.621792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:38283)
0807 04:26:46.622660 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:46.622932 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":391,"threads_started":1}
W20260807 04:26:46.624538 19366 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:45719 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:45719: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
I20260807 04:26:46.625403 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:46.625064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:42653)
0807 04:26:46.625216 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:46.625296 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 04:26:46.626477 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:46.626038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:56369)
0807 04:26:46.626208 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:46.626307 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:26:46.626747 19366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 04:26:46.628611 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19189
I20260807 04:26:46.629252 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:41865
--local_ip_for_outbound_sockets=127.1.244.194
--tserver_master_addrs=127.1.244.254:45719
--webserver_port=46669
--webserver_interface=127.1.244.194
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:26:46.629736 19254 raft_consensus.cc:348] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:46.630646 19254 raft_consensus.cc:729] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9370db9f44a94d17bb0b82e52dc71923, State: Initialized, Role: FOLLOWER
I20260807 04:26:46.631243 19254 consensus_queue.cc:260] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:46.632941 19254 ts_tablet_manager.cc:1428] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923: Time spent starting tablet: real 0.011s user 0.010s sys 0.002s
I20260807 04:26:46.633044 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:46.632563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:48567)
0807 04:26:46.632776 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:46.632863 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 04:26:46.633996 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:46.633576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:38731)
0807 04:26:46.633753 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:46.633839 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:26:46.958966 19370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:46.959282 19370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:46.959452 19370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:46.966482 19370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:46.966625 19370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:46.966812 19370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:26:46.976416 19370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:41865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=46669
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:46.977607 19370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:46.978895 19370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:46.992081 19378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:46.992136 19375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:46.992156 19376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:46.993180 19370 server_base.cc:1034] running on GCE node
I20260807 04:26:46.993851 19370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:46.994738 19370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:46.995991 19370 hybrid_clock.cc:648] HybridClock initialized: now 1786076806995942 us; error 66 us; skew 500 ppm
I20260807 04:26:46.996304 19370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:46 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:46.999284 19370 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:26:47.000732 19370 webserver.cc:460] Webserver started at http://127.1.244.194:46669/ using document root <none> and password file <none>
I20260807 04:26:47.001390 19370 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:47.001538 19370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:47.006649 19370 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 04:26:47.009299 19385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:47.010569 19370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:26:47.010746 19370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "05c863845e2f4937b0ca26c92f7e84af"
format_stamp: "Formatted at 2026-08-07 04:26:01 on dist-test-slave-zkpd"
server_key: "25ad531e5b35df05bc0b345381604325"
server_key_iv: "c264b1c4e37847dc57d5fa36b4e76dc5"
server_key_version: "encryptionkey@0"
I20260807 04:26:47.011768 19370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:47.025529 19370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:47.029348 19370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:47.029618 19370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:47.030544 19370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:47.031839 19370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:47.031998 19370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:47.032125 19370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:47.032215 19370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:47.065799 19370 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41865
I20260807 04:26:47.065887 19503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41865 every 8 connection(s)
I20260807 04:26:47.067241 19370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:26:47.067883 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19370
I20260807 04:26:47.068804 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:41425
--local_ip_for_outbound_sockets=127.1.244.195
--tserver_master_addrs=127.1.244.254:45719
--webserver_port=45855
--webserver_interface=127.1.244.195
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:26:47.072499 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.070649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:34553)
0807 04:26:47.071410 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.071615 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":306,"threads_started":1}
W20260807 04:26:47.073169 19504 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:45719 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:45719: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
I20260807 04:26:47.074023 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.073707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:55181)
0807 04:26:47.073863 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.073926 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 04:26:47.074834 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.074507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:58141)
0807 04:26:47.074644 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.074722 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 04:26:47.075064 19504 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 04:26:47.412906 19507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:26:47.413158 19507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:26:47.413367 19507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:26:47.420704 19507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:26:47.420836 19507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:26:47.421027 19507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:26:47.430608 19507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:41425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=45855
--tserver_master_addrs=127.1.244.254:45719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:26:47.431825 19507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:26:47.433136 19507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:26:47.446120 19513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:47.446147 19516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:26:47.446187 19514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:26:47.447145 19507 server_base.cc:1034] running on GCE node
I20260807 04:26:47.447751 19507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:26:47.448697 19507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:26:47.449896 19507 hybrid_clock.cc:648] HybridClock initialized: now 1786076807449855 us; error 41 us; skew 500 ppm
I20260807 04:26:47.450160 19507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:26:47 dist-test-slave-zkpd krb5kdc[17994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076807, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:26:47.453063 19507 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:26:47.454375 19507 webserver.cc:460] Webserver started at http://127.1.244.195:45855/ using document root <none> and password file <none>
I20260807 04:26:47.455029 19507 fs_manager.cc:359] Metadata directory not provided
I20260807 04:26:47.455157 19507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:26:47.461427 19507 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:26:47.465224 19524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:47.466814 19507 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 04:26:47.467007 19507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "f76fbebe5a8e4ac78b53ff400359ad07"
format_stamp: "Formatted at 2026-08-07 04:26:02 on dist-test-slave-zkpd"
server_key: "9fb216e7dd558abd70a459555e4c48e4"
server_key_iv: "d3a39fc4d2103f7dbdd72ef4508139a7"
server_key_version: "encryptionkey@0"
I20260807 04:26:47.468434 19507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:26:47.497303 19507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:26:47.501157 19507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:26:47.501442 19507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:26:47.502362 19507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:26:47.503638 19507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:26:47.503754 19507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:26:47.503888 19507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:26:47.503997 19507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 04:26:47.537346 19507 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:41425
I20260807 04:26:47.537442 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:41425 every 8 connection(s)
I20260807 04:26:47.538832 19507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:26:47.543607 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.541717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:55641)
0807 04:26:47.542480 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.542708 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":298,"threads_started":1}
W20260807 04:26:47.544374 19644 heartbeater.cc:644] Failed to heartbeat to 127.1.244.254:45719 (0 consecutive failures): Network error: Failed to ping master at 127.1.244.254:45719: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
I20260807 04:26:47.545149 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.544856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:59021)
0807 04:26:47.544983 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.545056 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 04:26:47.545924 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.545643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:55571)
0807 04:26:47.545753 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.545821 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 04:26:47.546160 19644 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 04:26:47.548760 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19507
I20260807 04:26:47.635457 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:47.634720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:52935)
0807 04:26:47.635172 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:47.635262 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":219,"threads_started":1}
I20260807 04:26:48.076550 19650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.075585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:40519)
0807 04:26:48.076190 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:48.076332 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":289,"threads_started":1}
I20260807 04:26:48.219082 19651 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:48.219375 19651 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:48.220963 19651 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
Aug 07 04:26:48 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
Aug 07 04:26:48 dist-test-slave-zkpd krb5kdc[17994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076806, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:26:48.257395 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.221323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41865 (local address 127.1.244.193:54437)
0807 04:26:48.234521 (+ 13198us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:48.234537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:48.234841 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:48.238311 (+ 3470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:48.238323 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:48.239548 (+ 1225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:48.240872 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:48.240901 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.243178 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.243184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:48.244884 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:48.244899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.245846 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.245852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:48.246118 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:48.246846 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:48.246879 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:48.249611 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:48.253899 (+ 4288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:48.253912 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:48.253922 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:48.254334 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:48.254817 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:48.254821 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:48.254827 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:48.255021 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:48.255832 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:48.255840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:48.256177 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 04:26:48.256454 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 04:26:48.256915 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13087,"mutex_wait_us":319,"spinlock_wait_cycles":4992,"thread_start_us":12796,"threads_started":1}
I20260807 04:26:48.257443 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.221642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:54437 (local address 127.1.244.194:41865)
0807 04:26:48.234270 (+ 12628us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:48.234286 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:48.234633 (+ 347us) server_negotiation.cc:407] Connection header received
0807 04:26:48.237389 (+ 2756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:48.237399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:48.237784 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:48.238146 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:48.241095 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.242965 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.245030 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.245646 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.245845 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:48.249903 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:48.249910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:48.249923 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:48.249958 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:48.253758 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:48.254485 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:48.254496 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:48.254506 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:48.254696 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:48.255145 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:48.255150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:48.255154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:48.255694 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:48.255942 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:48.256676 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 04:26:48.257060 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12396,"thread_start_us":12637,"threads_started":1}
I20260807 04:26:48.257640 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.221606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:50207 (local address 127.1.244.195:41425)
0807 04:26:48.233801 (+ 12195us) server_negotiation.cc:206] Beginning negotiation
0807 04:26:48.233821 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 04:26:48.234512 (+ 691us) server_negotiation.cc:407] Connection header received
0807 04:26:48.237281 (+ 2769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:26:48.237293 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:26:48.237785 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:26:48.238146 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:26:48.241095 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.242965 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.246077 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.247071 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.247254 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:48.250606 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:26:48.250614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:26:48.250626 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:26:48.250693 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:26:48.254444 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:48.255067 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:48.255075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:48.255082 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:48.255243 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:26:48.255663 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:26:48.255668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:26:48.255671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:26:48.256193 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:26:48.256396 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 04:26:48.256855 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 04:26:48.257245 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11988,"thread_start_us":11854,"threads_started":1}
I20260807 04:26:48.258025 19652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.221306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:41425 (local address 127.1.244.193:50207)
0807 04:26:48.234154 (+ 12848us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:48.234210 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 04:26:48.234841 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:26:48.238311 (+ 3470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:26:48.238323 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:26:48.239549 (+ 1226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:26:48.240872 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:48.240901 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.243182 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.243186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:48.244880 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:26:48.244893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:26:48.247271 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:26:48.247275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:26:48.247403 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:26:48.248060 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:26:48.248077 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:26:48.250279 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:26:48.254567 (+ 4288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:48.254570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:48.254573 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:48.254925 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:48.255396 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:26:48.255400 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:26:48.255402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:26:48.255541 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:26:48.256323 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:26:48.256328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:26:48.256486 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 04:26:48.256623 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 04:26:48.257125 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12694,"mutex_wait_us":428,"spinlock_wait_cycles":50944,"thread_start_us":12501,"threads_started":1}
W20260807 04:26:48.261281 19250 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
W20260807 04:26:48.261555 19249 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:48.261729 19249 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923; no voters: 05c863845e2f4937b0ca26c92f7e84af, f76fbebe5a8e4ac78b53ff400359ad07
I20260807 04:26:48.262163 19651 raft_consensus.cc:2726] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:26:48.547600 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.546714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:43007)
0807 04:26:48.547207 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:48.547342 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260807 04:26:48.636757 19652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:48.636352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:40735)
0807 04:26:48.636518 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:48.636618 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 04:26:49.077999 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:49.077358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:40969)
0807 04:26:49.077773 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:49.077852 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":217,"threads_started":1}
I20260807 04:26:49.549129 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:49.548485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:42417)
0807 04:26:49.548898 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:49.548981 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":211,"threads_started":1}
I20260807 04:26:49.638509 19659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:49.637676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:37553)
0807 04:26:49.638252 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:49.638332 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":308,"threads_started":1}
I20260807 04:26:50.079551 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:50.078887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:55889)
0807 04:26:50.079309 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:50.079390 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":228,"threads_started":1}
I20260807 04:26:50.305595 19661 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:50.305814 19661 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:50.306483 19661 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
W20260807 04:26:50.307397 19250 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
W20260807 04:26:50.307605 19249 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:50.307740 19249 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923; no voters: 05c863845e2f4937b0ca26c92f7e84af, f76fbebe5a8e4ac78b53ff400359ad07
I20260807 04:26:50.308039 19661 raft_consensus.cc:2726] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:26:50.550647 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:50.549951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:60713)
0807 04:26:50.550401 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:50.550492 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":219,"threads_started":1}
I20260807 04:26:50.640290 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:50.639446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:34707)
0807 04:26:50.639971 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:50.640098 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260807 04:26:51.081065 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:51.080403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:33711)
0807 04:26:51.080809 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:51.080889 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":200,"threads_started":1}
I20260807 04:26:51.552461 19665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:51.551552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:40853)
0807 04:26:51.552180 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:51.552277 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":300,"threads_started":1}
I20260807 04:26:51.642345 19666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:51.641386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:33699)
0807 04:26:51.642006 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:51.642109 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":328,"threads_started":1}
I20260807 04:26:52.082731 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:52.082079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:40305)
0807 04:26:52.082496 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:52.082573 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":233,"threads_started":1}
I20260807 04:26:52.554064 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:52.553391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:52751)
0807 04:26:52.553823 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:52.553914 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":213,"threads_started":1}
I20260807 04:26:52.644032 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:52.643193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:43097)
0807 04:26:52.643683 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:52.643765 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":264,"threads_started":1}
I20260807 04:26:52.937290 19670 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:52.937515 19670 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:52.938210 19670 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
W20260807 04:26:52.939179 19250 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
W20260807 04:26:52.939364 19249 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:52.939468 19249 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923; no voters: 05c863845e2f4937b0ca26c92f7e84af, f76fbebe5a8e4ac78b53ff400359ad07
I20260807 04:26:52.939687 19670 raft_consensus.cc:2726] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:26:53.084298 19671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:53.083526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:36473)
0807 04:26:53.084047 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:53.084126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":255,"threads_started":1}
I20260807 04:26:53.555555 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:53.554874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:48975)
0807 04:26:53.555302 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:53.555379 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
I20260807 04:26:53.645596 19674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:53.644884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:58241)
0807 04:26:53.645303 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:53.645394 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":225,"threads_started":1}
I20260807 04:26:54.086040 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:54.085188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:60959)
0807 04:26:54.085740 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:54.085849 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":237,"threads_started":1}
I20260807 04:26:54.557077 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:54.556379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:41931)
0807 04:26:54.556817 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:54.556891 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":201,"threads_started":1}
I20260807 04:26:54.610872 19677 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:54.611101 19677 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:54.611759 19677 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
W20260807 04:26:54.612677 19250 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
W20260807 04:26:54.612871 19249 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:54.612967 19249 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923; no voters: 05c863845e2f4937b0ca26c92f7e84af, f76fbebe5a8e4ac78b53ff400359ad07
I20260807 04:26:54.613181 19677 raft_consensus.cc:2726] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:26:54.647357 19678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:54.646555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:32865)
0807 04:26:54.647054 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:54.647152 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":289,"threads_started":1}
I20260807 04:26:55.087632 19679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:55.086986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:41643)
0807 04:26:55.087371 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:55.087476 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":202,"threads_started":1}
I20260807 04:26:55.558771 19680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:55.558096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:55889)
0807 04:26:55.558493 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:55.558579 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":215,"threads_started":1}
I20260807 04:26:55.648898 19681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:55.648178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:47071)
0807 04:26:55.648623 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:55.648704 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
I20260807 04:26:56.089226 19682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:56.088515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:58491)
0807 04:26:56.088992 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:56.089078 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":227,"threads_started":1}
I20260807 04:26:56.560423 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:56.559610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:44709)
0807 04:26:56.560174 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:56.560258 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":288,"threads_started":1}
I20260807 04:26:56.650605 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:56.649867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:58497)
0807 04:26:56.650307 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:56.650434 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":251,"threads_started":1}
I20260807 04:26:57.090799 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:57.090153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:38183)
0807 04:26:57.090557 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:57.090635 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":209,"threads_started":1}
I20260807 04:26:57.562054 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:57.561317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:52877)
0807 04:26:57.561790 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:57.561879 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":238,"threads_started":1}
I20260807 04:26:57.652182 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:57.651428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:41275)
0807 04:26:57.651860 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:57.652003 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":211,"threads_started":1}
I20260807 04:26:58.068835 19688 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:26:58.069051 19688 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:26:58.069795 19688 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
W20260807 04:26:58.070752 19250 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
W20260807 04:26:58.070962 19249 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:26:58.071065 19249 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923; no voters: 05c863845e2f4937b0ca26c92f7e84af, f76fbebe5a8e4ac78b53ff400359ad07
I20260807 04:26:58.071327 19688 raft_consensus.cc:2726] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:26:58.092406 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:58.091663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:42425)
0807 04:26:58.092157 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:58.092242 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260807 04:26:58.563694 19690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:58.563017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:40279)
0807 04:26:58.563449 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:58.563542 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":209,"threads_started":1}
I20260807 04:26:58.653646 19691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:58.652982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:42669)
0807 04:26:58.653411 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:58.653495 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":214,"threads_started":1}
I20260807 04:26:59.094038 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:59.093238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:54511)
0807 04:26:59.093783 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:59.093865 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":308,"threads_started":1}
I20260807 04:26:59.565321 19693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:59.564599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:38037)
0807 04:26:59.565058 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:59.565149 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":268,"threads_started":1}
I20260807 04:26:59.655299 19694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:26:59.654590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:39555)
0807 04:26:59.655034 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 04:26:59.655127 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260807 04:27:00.095582 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:00.094859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:46067)
0807 04:27:00.095331 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:00.095427 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":237,"threads_started":1}
I20260807 04:27:00.566903 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:00.566240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:40639)
0807 04:27:00.566657 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:00.566741 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":217,"threads_started":1}
I20260807 04:27:00.657183 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:00.656400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:38271)
0807 04:27:00.656911 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:00.656992 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":238,"threads_started":1}
I20260807 04:27:01.097205 19698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:01.096436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:46991)
0807 04:27:01.096918 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:01.097011 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":247,"threads_started":1}
I20260807 04:27:01.568622 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:01.567773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:45565)
0807 04:27:01.568337 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:01.568431 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":361,"threads_started":1}
I20260807 04:27:01.659085 19700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:01.658319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.193:43085)
0807 04:27:01.658834 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:01.658915 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":220,"threads_started":1}
I20260807 04:27:01.906313 19701 raft_consensus.cc:482] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:01.906550 19701 raft_consensus.cc:504] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76fbebe5a8e4ac78b53ff400359ad07" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 41425 } } peers { permanent_uuid: "05c863845e2f4937b0ca26c92f7e84af" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41865 } } peers { permanent_uuid: "9370db9f44a94d17bb0b82e52dc71923" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 32837 } }
I20260807 04:27:01.907230 19701 leader_election.cc:288] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425), 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865)
W20260807 04:27:01.908165 19249 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05c863845e2f4937b0ca26c92f7e84af (127.1.244.194:41865): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
W20260807 04:27:01.908340 19250 leader_election.cc:341] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f76fbebe5a8e4ac78b53ff400359ad07 (127.1.244.195:41425): Not found: Tablet not found: 057d4f8954d14ae0baafc1feccbd0c9c
I20260807 04:27:01.908506 19250 leader_election.cc:302] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9370db9f44a94d17bb0b82e52dc71923; no voters: 05c863845e2f4937b0ca26c92f7e84af, f76fbebe5a8e4ac78b53ff400359ad07
I20260807 04:27:01.908763 19701 raft_consensus.cc:2726] T 057d4f8954d14ae0baafc1feccbd0c9c P 9370db9f44a94d17bb0b82e52dc71923 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 04:27:02.098929 19702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:02.098208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.194:49603)
0807 04:27:02.098661 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:02.098775 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":212,"threads_started":1}
I20260807 04:27:02.555716 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 19189
I20260807 04:27:02.570832 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:02.569863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:45719 (local address 127.1.244.195:44667)
0807 04:27:02.570556 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:02.570671 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.254:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":336,"threads_started":1}
I20260807 04:27:02.571702 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:02.556684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47472 (local address 127.1.244.193:32837)
0807 04:27:02.557552 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:02.557560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:02.557587 (+ 27us) server_negotiation.cc:407] Connection header received
0807 04:27:02.557680 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:02.557686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:02.557906 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:02.558024 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:02.559590 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:02.560690 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:02.561907 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:02.562453 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:02.562567 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:02.565228 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:02.565239 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:02.565252 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:02.565285 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:02.568253 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:02.568941 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:02.568947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:02.568950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:02.569148 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:02.569585 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:02.569610 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:02.569613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:02.570148 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:02.570464 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:02.570984 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 04:27:02.571268 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":242,"threads_started":1}
I20260807 04:27:02.573012 19361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:02.661525 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 19189
I20260807 04:27:02.682436 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 19370
I20260807 04:27:02.695513 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:02.683159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.1.244.194:41865)
0807 04:27:02.683676 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:02.683680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:02.683694 (+ 14us) server_negotiation.cc:407] Connection header received
0807 04:27:02.683748 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:02.683752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:02.683836 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:02.684023 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:02.685365 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:02.686116 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:02.687257 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:02.687653 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:02.687798 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:02.690463 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:02.690468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:02.690471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:02.690502 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:02.692893 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:02.693546 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:02.693556 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:02.693558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:02.693693 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:02.694055 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:02.694061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:02.694064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:02.694333 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:02.694595 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:02.694889 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 04:27:02.695117 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":281,"threads_started":1}
I20260807 04:27:02.696269 19498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:02.779263 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 19370
I20260807 04:27:02.798866 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 19507
I20260807 04:27:02.812191 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:02.799699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.1.244.195:41425)
0807 04:27:02.800313 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:02.800318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:02.800335 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:27:02.800392 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:02.800395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:02.800458 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:02.800550 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:02.802062 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:02.802859 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:02.804212 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:02.804618 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:02.804713 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:02.807220 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:02.807225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:02.807228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:02.807258 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:02.809792 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:02.810387 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:02.810391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:02.810393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:02.810523 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:02.810841 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:02.810844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:02.810846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:02.811102 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:02.811300 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:02.811504 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 04:27:02.811660 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":342,"threads_started":1}
I20260807 04:27:02.812848 19639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:02.895151 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 19507
2026-08-07T04:27:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64272 ms)
[----------] 30 tests from SecurityITest (296400 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:03 dist-test-slave-zkpd krb5kdc[19715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:03 dist-test-slave-zkpd krb5kdc[19715](info): set up 2 sockets
Aug 07 04:27:03 dist-test-slave-zkpd krb5kdc[19715](info): commencing operation
krb5kdc: starting...
W20260807 04:27:04.027490 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:04 dist-test-slave-zkpd krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:04.307137 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33105
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:33105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:04.642042 19730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:04.642313 19730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:04.642545 19730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:04.649719 19730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:04.649842 19730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:04.649951 19730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:04.650060 19730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:04.650144 19730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:04.659480 19730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:33105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:33105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:04.660739 19730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:04.662042 19730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:04.674225 19738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:04.674355 19735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:04.674261 19736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:04.675199 19730 server_base.cc:1034] running on GCE node
I20260807 04:27:04.675945 19730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:04.677263 19730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:04.678501 19730 hybrid_clock.cc:648] HybridClock initialized: now 1786076824678459 us; error 47 us; skew 500 ppm
I20260807 04:27:04.678829 19730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:04 dist-test-slave-zkpd krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076824, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:04.686885 19730 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:04.688501 19730 webserver.cc:460] Webserver started at http://127.1.244.254:45181/ using document root <none> and password file <none>
I20260807 04:27:04.689194 19730 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:04.689342 19730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:04.689659 19730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:04.692690 19730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "a2c8036c238449438ea30315daa3f34f"
format_stamp: "Formatted at 2026-08-07 04:27:04 on dist-test-slave-zkpd"
server_key: "ca68460b8a6cde1ef5e88ed4188c884a"
server_key_iv: "aa6b07691bb05b35e7b7dc745a3938db"
server_key_version: "encryptionkey@0"
I20260807 04:27:04.693463 19730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "a2c8036c238449438ea30315daa3f34f"
format_stamp: "Formatted at 2026-08-07 04:27:04 on dist-test-slave-zkpd"
server_key: "ca68460b8a6cde1ef5e88ed4188c884a"
server_key_iv: "aa6b07691bb05b35e7b7dc745a3938db"
server_key_version: "encryptionkey@0"
I20260807 04:27:04.698544 19730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:04.702097 19746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:04.703372 19730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 04:27:04.703557 19730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "a2c8036c238449438ea30315daa3f34f"
format_stamp: "Formatted at 2026-08-07 04:27:04 on dist-test-slave-zkpd"
server_key: "ca68460b8a6cde1ef5e88ed4188c884a"
server_key_iv: "aa6b07691bb05b35e7b7dc745a3938db"
server_key_version: "encryptionkey@0"
I20260807 04:27:04.703753 19730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:04.751945 19730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:04.756001 19730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:04.756299 19730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:04.774802 19730 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:33105
I20260807 04:27:04.774843 19799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:33105 every 8 connection(s)
I20260807 04:27:04.776175 19730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:04.779693 19800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:04.785163 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19730
I20260807 04:27:04.785416 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:04.786324 2003 external_mini_cluster.cc:1428] Setting key e0426c21a046f434dfc2a4fe32a6a260
I20260807 04:27:04.790762 19800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f: Bootstrap starting.
I20260807 04:27:04.794420 19800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:04.795565 19800 log.cc:824] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:04.798494 19800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f: No bootstrap required, opened a new log
Aug 07 04:27:04 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:04.806885 19800 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c8036c238449438ea30315daa3f34f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33105 } }
I20260807 04:27:04.807256 19800 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:04.807396 19800 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c8036c238449438ea30315daa3f34f, State: Initialized, Role: FOLLOWER
I20260807 04:27:04.808079 19800 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c8036c238449438ea30315daa3f34f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33105 } }
I20260807 04:27:04.808485 19800 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:04.808638 19800 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:04.808844 19800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:04.809448 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:04.788454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.1.244.254:33105)
0807 04:27:04.789270 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:04.789281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:04.789322 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:27:04.791021 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:04.791043 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:04.791386 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:04.791675 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:04.793654 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:04.795005 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:04.796796 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:04.797434 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:04.797582 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:04.802259 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:04.802266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:04.802282 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:04.802323 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:04.805303 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:04.806058 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:04.806065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:04.806070 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:04.806238 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:04.806630 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:04.806635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:04.806637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:04.807125 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:04.807497 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:04.807941 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 04:27:04.808304 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":403,"threads_started":1}
I20260807 04:27:04.811208 19800 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c8036c238449438ea30315daa3f34f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33105 } }
I20260807 04:27:04.811689 19800 leader_election.cc:302] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2c8036c238449438ea30315daa3f34f; no voters:
I20260807 04:27:04.812367 19800 leader_election.cc:288] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:04.812455 19805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:04.813220 19805 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [term 1 LEADER]: Becoming Leader. State: Replica: a2c8036c238449438ea30315daa3f34f, State: Running, Role: LEADER
I20260807 04:27:04.813807 19805 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c8036c238449438ea30315daa3f34f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33105 } }
I20260807 04:27:04.814481 19800 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:04.816577 19807 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c8036c238449438ea30315daa3f34f. Latest consensus state: current_term: 1 leader_uuid: "a2c8036c238449438ea30315daa3f34f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c8036c238449438ea30315daa3f34f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33105 } } }
I20260807 04:27:04.817045 19807 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:04.818195 19806 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2c8036c238449438ea30315daa3f34f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c8036c238449438ea30315daa3f34f" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 33105 } } }
I20260807 04:27:04.818638 19806 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:04.823006 19814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:04.827982 19814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:04.836310 19814 catalog_manager.cc:1347] Generated new cluster ID: 180d038530424fd1a1b8378cf334fa63
I20260807 04:27:04.836426 19814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:04.854651 19814 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:04.856457 19814 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:04.868021 19814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f: Generated new TSK 0
I20260807 04:27:04.868974 19814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:27:04.937551 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33105
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:05.274055 19828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:05.274307 19828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:05.274489 19828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:05.281549 19828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:05.281678 19828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:05.281837 19828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:27:05.291375 19828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:05.292615 19828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:05.293947 19828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:05.307154 19836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:05.307130 19834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:05.307224 19833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:05.307916 19828 server_base.cc:1034] running on GCE node
I20260807 04:27:05.308496 19828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:05.309424 19828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:05.310655 19828 hybrid_clock.cc:648] HybridClock initialized: now 1786076825310624 us; error 51 us; skew 500 ppm
I20260807 04:27:05.310972 19828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:05 dist-test-slave-zkpd krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076825, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:05.314033 19828 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:27:05.315544 19828 webserver.cc:460] Webserver started at http://127.1.244.193:42535/ using document root <none> and password file <none>
I20260807 04:27:05.316275 19828 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:05.316423 19828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:05.316727 19828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:05.319552 19828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "7c0adfc8951f41669f4382a193e4f262"
format_stamp: "Formatted at 2026-08-07 04:27:05 on dist-test-slave-zkpd"
server_key: "926dd220d91fbe817ff86f94a8e58cfa"
server_key_iv: "dadcccd759c2873809554f20f265c4b1"
server_key_version: "encryptionkey@0"
I20260807 04:27:05.320442 19828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "7c0adfc8951f41669f4382a193e4f262"
format_stamp: "Formatted at 2026-08-07 04:27:05 on dist-test-slave-zkpd"
server_key: "926dd220d91fbe817ff86f94a8e58cfa"
server_key_iv: "dadcccd759c2873809554f20f265c4b1"
server_key_version: "encryptionkey@0"
I20260807 04:27:05.325515 19828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 04:27:05.328980 19843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:05.330289 19828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:05.330466 19828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "7c0adfc8951f41669f4382a193e4f262"
format_stamp: "Formatted at 2026-08-07 04:27:05 on dist-test-slave-zkpd"
server_key: "926dd220d91fbe817ff86f94a8e58cfa"
server_key_iv: "dadcccd759c2873809554f20f265c4b1"
server_key_version: "encryptionkey@0"
I20260807 04:27:05.330672 19828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:05.349256 19828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:05.353045 19828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:05.353314 19828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:05.354220 19828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:05.355472 19828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:05.355546 19828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:05.355710 19828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:05.355762 19828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:05.387653 19828 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:40615
I20260807 04:27:05.387696 19961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:40615 every 8 connection(s)
I20260807 04:27:05.388933 19828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:27:05.393988 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19828
I20260807 04:27:05.394207 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:27:05.394845 2003 external_mini_cluster.cc:1428] Setting key b847f80af33594ab55d245be82cfa6d0
Aug 07 04:27:05 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076825, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:05.409186 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:05.391504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:58713 (local address 127.1.244.254:33105)
0807 04:27:05.392022 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:05.392026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:05.392891 (+ 865us) server_negotiation.cc:407] Connection header received
0807 04:27:05.394893 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:05.394898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:05.395002 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:05.395123 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:05.396740 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.397501 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.399076 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.399503 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.399643 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:05.403167 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:05.403172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:05.403175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:05.403208 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:05.405854 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:05.406644 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:05.406656 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:05.406659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:05.406821 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:05.407274 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:05.407279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:05.407282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:05.407570 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:05.407728 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:05.408582 (+ 854us) server_negotiation.cc:299] Negotiation successful
0807 04:27:05.408809 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":252,"threads_started":1}
I20260807 04:27:05.410116 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:05.391961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33105 (local address 127.1.244.193:58713)
0807 04:27:05.392717 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:05.392766 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:05.394619 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:05.395394 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:05.395402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:05.396006 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:05.396563 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:05.396580 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.397678 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.397682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:05.398925 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:05.398937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.399758 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.399765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:05.400060 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:05.400755 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:05.400782 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:05.402935 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:05.405999 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:05.406009 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:05.406025 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:05.406515 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:05.406953 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:05.406960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:05.406968 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:05.407150 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:05.407702 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:05.407709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:05.408202 (+ 493us) client_negotiation.cc:769] Sending connection context
0807 04:27:05.408475 (+ 273us) client_negotiation.cc:240] Negotiation successful
0807 04:27:05.408990 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":259,"threads_started":1}
I20260807 04:27:05.411715 19963 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33105
I20260807 04:27:05.412043 19963 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:05.412747 19963 heartbeater.cc:511] Master 127.1.244.254:33105 requested a full tablet report, sending...
I20260807 04:27:05.415098 19763 ts_manager.cc:194] Registered new tserver with Master: 7c0adfc8951f41669f4382a193e4f262 (127.1.244.193:40615)
I20260807 04:27:05.417291 19763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:58713
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:27:05.458339 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33105
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:05.789047 19969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:05.789294 19969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:05.789479 19969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:05.796674 19969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:05.796806 19969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:05.796990 19969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:27:05.806452 19969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:05.807700 19969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:05.809062 19969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:05.821806 19974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:05.821890 19975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:05.822255 19977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:05.823158 19969 server_base.cc:1034] running on GCE node
I20260807 04:27:05.823743 19969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:05.824715 19969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:05.825966 19969 hybrid_clock.cc:648] HybridClock initialized: now 1786076825825903 us; error 51 us; skew 500 ppm
I20260807 04:27:05.826268 19969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:05 dist-test-slave-zkpd krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076825, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:05.829380 19969 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:27:05.830677 19969 webserver.cc:460] Webserver started at http://127.1.244.194:46729/ using document root <none> and password file <none>
I20260807 04:27:05.831269 19969 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:05.831367 19969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:05.831624 19969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:05.834522 19969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "0ac57887aaaa44b2877127c5e2afc516"
format_stamp: "Formatted at 2026-08-07 04:27:05 on dist-test-slave-zkpd"
server_key: "1f4bd0b9df23fd6c96e009a1c5010f25"
server_key_iv: "f3688b1b802e46777f430d62b74698c1"
server_key_version: "encryptionkey@0"
I20260807 04:27:05.835368 19969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "0ac57887aaaa44b2877127c5e2afc516"
format_stamp: "Formatted at 2026-08-07 04:27:05 on dist-test-slave-zkpd"
server_key: "1f4bd0b9df23fd6c96e009a1c5010f25"
server_key_iv: "f3688b1b802e46777f430d62b74698c1"
server_key_version: "encryptionkey@0"
I20260807 04:27:05.840291 19969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 04:27:05.843516 19984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:05.844739 19969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:05.844900 19969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "0ac57887aaaa44b2877127c5e2afc516"
format_stamp: "Formatted at 2026-08-07 04:27:05 on dist-test-slave-zkpd"
server_key: "1f4bd0b9df23fd6c96e009a1c5010f25"
server_key_iv: "f3688b1b802e46777f430d62b74698c1"
server_key_version: "encryptionkey@0"
I20260807 04:27:05.845098 19969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:05.858999 19969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:05.862612 19969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:05.862841 19969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:05.863703 19969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:05.864964 19969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:05.865037 19969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:05.865119 19969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:05.865166 19969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:05.897886 19969 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:43643
I20260807 04:27:05.897964 20101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:43643 every 8 connection(s)
I20260807 04:27:05.899152 19969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:27:05.906311 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 19969
I20260807 04:27:05.906598 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:27:05.907176 2003 external_mini_cluster.cc:1428] Setting key 3561fa93f509d746bcca238bef2b250f
Aug 07 04:27:05 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076825, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:05.919658 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:05.901615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:41117 (local address 127.1.244.254:33105)
0807 04:27:05.901829 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:05.901833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:05.903034 (+ 1201us) server_negotiation.cc:407] Connection header received
0807 04:27:05.904979 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:05.904985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:05.905097 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:05.905212 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:05.906698 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.907547 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.909490 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.909918 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.910051 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:05.913967 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:05.913972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:05.913975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:05.914008 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:05.916679 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:05.917378 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:05.917388 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:05.917392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:05.917528 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:05.917993 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:05.917996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:05.917999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:05.918243 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:05.918401 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:05.919077 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 04:27:05.919299 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:27:05.920665 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:05.902162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33105 (local address 127.1.244.194:41117)
0807 04:27:05.902866 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:05.902903 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:05.904683 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:05.905454 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:05.905462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:05.905964 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:05.906498 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:05.906515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.907722 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.907732 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:05.909306 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:05.909319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:05.910146 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:05.910154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:05.910372 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:05.911118 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:05.911151 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:05.913741 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:05.916820 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:05.916828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:05.916841 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:05.917231 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:05.917660 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:05.917665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:05.917669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:05.917860 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:05.918373 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:05.918380 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:05.918707 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 04:27:05.918970 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 04:27:05.919490 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":295,"threads_started":1}
I20260807 04:27:05.922220 20102 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33105
I20260807 04:27:05.922539 20102 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:05.923353 20102 heartbeater.cc:511] Master 127.1.244.254:33105 requested a full tablet report, sending...
I20260807 04:27:05.924989 19763 ts_manager.cc:194] Registered new tserver with Master: 0ac57887aaaa44b2877127c5e2afc516 (127.1.244.194:43643)
I20260807 04:27:05.926239 19763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:41117
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:27:05.970759 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33105
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:06.303027 20108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:06.303282 20108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:06.303371 20108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:06.310405 20108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:06.310500 20108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:06.310614 20108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:27:06.319870 20108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:33105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:06.321054 20108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:06.322311 20108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:06.335579 20113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:06.335551 20116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:06.335624 20114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:06.336644 20108 server_base.cc:1034] running on GCE node
I20260807 04:27:06.337353 20108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:06.338251 20108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:06.339444 20108 hybrid_clock.cc:648] HybridClock initialized: now 1786076826339396 us; error 37 us; skew 500 ppm
I20260807 04:27:06.339763 20108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:06 dist-test-slave-zkpd krb5kdc[19715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076826, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:06.342896 20108 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:27:06.344355 20108 webserver.cc:460] Webserver started at http://127.1.244.195:39837/ using document root <none> and password file <none>
I20260807 04:27:06.344987 20108 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:06.345118 20108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:06.345431 20108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:06.348322 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "3ca43aec4ade4626b571270e98d29dae"
format_stamp: "Formatted at 2026-08-07 04:27:06 on dist-test-slave-zkpd"
server_key: "a2118fe263e2ed284c936fc95197abf6"
server_key_iv: "ba62c85a81b9e1456ec25d0579c3da6e"
server_key_version: "encryptionkey@0"
I20260807 04:27:06.349079 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "3ca43aec4ade4626b571270e98d29dae"
format_stamp: "Formatted at 2026-08-07 04:27:06 on dist-test-slave-zkpd"
server_key: "a2118fe263e2ed284c936fc95197abf6"
server_key_iv: "ba62c85a81b9e1456ec25d0579c3da6e"
server_key_version: "encryptionkey@0"
I20260807 04:27:06.354123 20108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 04:27:06.357460 20123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:06.358621 20108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 04:27:06.358783 20108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "3ca43aec4ade4626b571270e98d29dae"
format_stamp: "Formatted at 2026-08-07 04:27:06 on dist-test-slave-zkpd"
server_key: "a2118fe263e2ed284c936fc95197abf6"
server_key_iv: "ba62c85a81b9e1456ec25d0579c3da6e"
server_key_version: "encryptionkey@0"
I20260807 04:27:06.358986 20108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:06.372913 20108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:06.377146 20108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:06.377430 20108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:06.378332 20108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:06.379532 20108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:06.379602 20108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:06.379765 20108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:06.379843 20108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:06.413164 20108 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:38743
I20260807 04:27:06.413203 20241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:38743 every 8 connection(s)
I20260807 04:27:06.414484 20108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:27:06.420151 19963 heartbeater.cc:503] Master 127.1.244.254:33105 was elected leader, sending a full tablet report...
I20260807 04:27:06.424839 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20108
I20260807 04:27:06.425025 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:27:06.425467 2003 external_mini_cluster.cc:1428] Setting key 883ba5c849c8c70266b945e37bbd81dc
Aug 07 04:27:06 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076826, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:06.434708 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.417204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:58965 (local address 127.1.244.254:33105)
0807 04:27:06.417426 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.417432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.418486 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 04:27:06.420327 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.420333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.420441 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.420589 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:06.422343 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.423198 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.424799 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.425369 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.425506 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.429334 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:06.429345 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:06.429349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:06.429379 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:06.431731 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.432460 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.432465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.432469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.432608 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.433042 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.433047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.433049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.433300 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:06.433458 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.434133 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.434357 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:27:06.435573 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.417641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:33105 (local address 127.1.244.195:58965)
0807 04:27:06.418334 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:06.418375 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:06.420056 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:06.420814 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:06.420822 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:06.421597 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:06.422183 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.422200 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.423385 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.423389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.424645 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.424657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.425626 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.425633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.425866 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.426639 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:06.426672 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:06.429143 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:06.431911 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:06.431920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:06.431935 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:06.432337 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:06.432720 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:06.432725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:06.432729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:06.432918 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:06.433429 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:06.433437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:06.433771 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 04:27:06.434016 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 04:27:06.434540 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":270,"threads_started":1}
I20260807 04:27:06.436872 20242 heartbeater.cc:347] Connected to a master server at 127.1.244.254:33105
I20260807 04:27:06.437175 20242 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:06.437885 20242 heartbeater.cc:511] Master 127.1.244.254:33105 requested a full tablet report, sending...
I20260807 04:27:06.439410 19763 ts_manager.cc:194] Registered new tserver with Master: 3ca43aec4ade4626b571270e98d29dae (127.1.244.195:38743)
I20260807 04:27:06.440608 19763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:58965
I20260807 04:27:06.441149 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:27:06.458308 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.445581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.1.244.254:33105)
0807 04:27:06.445755 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.445763 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.446335 (+ 572us) server_negotiation.cc:407] Connection header received
0807 04:27:06.446524 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.446529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.446600 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.446699 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:06.448132 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.448899 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.450190 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.450752 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.450900 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.453225 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:06.453231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:06.453236 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:06.453291 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:06.455536 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.456254 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.456260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.456263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.456394 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.456796 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.456801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.456804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.457106 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:06.457387 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.457739 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.457955 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 04:27:06.461879 19763 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:27:06.468605 19763 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:27:06.505028 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.481106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52340 (local address 127.1.244.194:43643)
0807 04:27:06.494484 (+ 13378us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.494497 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.495614 (+ 1117us) server_negotiation.cc:407] Connection header received
0807 04:27:06.495745 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.495757 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.496715 (+ 958us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.496902 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:06.497775 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.499182 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.501973 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.503418 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.503574 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.504425 (+ 851us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.504498 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.504696 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13153,"thread_start_us":13396,"threads_started":1}
I20260807 04:27:06.505146 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.480554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:43643 (local address 127.0.0.1:52340)
0807 04:27:06.495247 (+ 14693us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:06.495270 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:06.495429 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:06.497164 (+ 1735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:06.497173 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:06.497204 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:06.497609 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.497622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.499320 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.499334 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.501771 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.501787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.504067 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.504074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.504270 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.504320 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 04:27:06.504442 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 04:27:06.504692 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14534,"mutex_wait_us":1,"thread_start_us":15992,"threads_started":1}
I20260807 04:27:06.505498 20254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.480387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:40615 (local address 127.0.0.1:57538)
0807 04:27:06.487487 (+ 7100us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:06.487542 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:06.487766 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:06.498234 (+ 10468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:06.498238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:06.498264 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:06.498690 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.498704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.500801 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.500807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.502731 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.502743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.504714 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.504720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.504909 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.504940 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:27:06.505008 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 04:27:06.505150 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6774,"mutex_wait_us":84,"thread_start_us":6616,"threads_started":1}
I20260807 04:27:06.506465 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.480628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.1.244.193:40615)
0807 04:27:06.497632 (+ 17004us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.497638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.497659 (+ 21us) server_negotiation.cc:407] Connection header received
0807 04:27:06.497765 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.497771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.498009 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.498805 (+ 796us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:06.499090 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.500641 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.502894 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.504459 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.505725 (+ 1266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.505937 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.506004 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.506157 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16859,"thread_start_us":16696,"threads_started":1}
I20260807 04:27:06.507083 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.493337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38743 (local address 127.0.0.1:54414)
0807 04:27:06.496877 (+ 3540us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:06.496899 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:06.497049 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:06.498058 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:06.498063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:06.498087 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:06.498516 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.498526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.500915 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.500922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.503030 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.503046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.505382 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.505396 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.506424 (+ 1028us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.506469 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 04:27:06.506563 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 04:27:06.506693 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3366,"mutex_wait_us":59,"thread_start_us":1549,"threads_started":1}
I20260807 04:27:06.507323 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.488426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.1.244.195:38743)
0807 04:27:06.496518 (+ 8092us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.496529 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.497566 (+ 1037us) server_negotiation.cc:407] Connection header received
0807 04:27:06.497674 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.497684 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.497870 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.498071 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:06.498707 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.500783 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.503224 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.504689 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.504819 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.505017 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.506757 (+ 1740us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.506964 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7876,"thread_start_us":7698,"threads_started":1}
I20260807 04:27:06.509413 20035 tablet_service.cc:1467] Processing CreateTablet for tablet 4938814b793744afa2212d12b63abca1 (DEFAULT_TABLE table=test-table [id=6ea21cb77cbf453aa4546392f3ab3193]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:06.510520 20173 tablet_service.cc:1467] Processing CreateTablet for tablet 4938814b793744afa2212d12b63abca1 (DEFAULT_TABLE table=test-table [id=6ea21cb77cbf453aa4546392f3ab3193]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:06.510963 20035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4938814b793744afa2212d12b63abca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:06.510460 19895 tablet_service.cc:1467] Processing CreateTablet for tablet 4938814b793744afa2212d12b63abca1 (DEFAULT_TABLE table=test-table [id=6ea21cb77cbf453aa4546392f3ab3193]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:06.511695 20173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4938814b793744afa2212d12b63abca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:06.511821 19895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4938814b793744afa2212d12b63abca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:06.524669 20260 tablet_bootstrap.cc:492] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Bootstrap starting.
I20260807 04:27:06.528630 20261 tablet_bootstrap.cc:492] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Bootstrap starting.
I20260807 04:27:06.529594 20260 tablet_bootstrap.cc:654] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:06.531098 20260 log.cc:824] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:06.531999 20262 tablet_bootstrap.cc:492] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Bootstrap starting.
I20260807 04:27:06.532311 20261 tablet_bootstrap.cc:654] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:06.533558 20261 log.cc:824] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:06.536572 20260 tablet_bootstrap.cc:492] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: No bootstrap required, opened a new log
I20260807 04:27:06.536871 20260 ts_tablet_manager.cc:1397] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 04:27:06.536895 20262 tablet_bootstrap.cc:654] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:06.537976 20262 log.cc:824] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:06.538589 20261 tablet_bootstrap.cc:492] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: No bootstrap required, opened a new log
I20260807 04:27:06.538836 20261 ts_tablet_manager.cc:1397] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 04:27:06.540452 20262 tablet_bootstrap.cc:492] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: No bootstrap required, opened a new log
I20260807 04:27:06.540689 20262 ts_tablet_manager.cc:1397] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 04:27:06.548075 20262 raft_consensus.cc:348] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.548105 20260 raft_consensus.cc:348] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.548419 20262 raft_consensus.cc:374] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:06.548516 20260 raft_consensus.cc:374] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:06.548513 20262 raft_consensus.cc:729] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac57887aaaa44b2877127c5e2afc516, State: Initialized, Role: FOLLOWER
I20260807 04:27:06.548653 20260 raft_consensus.cc:729] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca43aec4ade4626b571270e98d29dae, State: Initialized, Role: FOLLOWER
I20260807 04:27:06.549122 20262 consensus_queue.cc:260] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.549242 20260 consensus_queue.cc:260] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.549537 20261 raft_consensus.cc:348] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.550019 20261 raft_consensus.cc:374] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:06.550124 20261 raft_consensus.cc:729] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0adfc8951f41669f4382a193e4f262, State: Initialized, Role: FOLLOWER
I20260807 04:27:06.550753 20242 heartbeater.cc:503] Master 127.1.244.254:33105 was elected leader, sending a full tablet report...
I20260807 04:27:06.551222 20260 ts_tablet_manager.cc:1428] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260807 04:27:06.551613 20261 consensus_queue.cc:260] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.555179 20261 ts_tablet_manager.cc:1428] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260807 04:27:06.556175 20102 heartbeater.cc:503] Master 127.1.244.254:33105 was elected leader, sending a full tablet report...
I20260807 04:27:06.558264 20262 ts_tablet_manager.cc:1428] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260807 04:27:06.573559 20268 raft_consensus.cc:482] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:06.573876 20268 raft_consensus.cc:504] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.575398 20268 leader_election.cc:288] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0adfc8951f41669f4382a193e4f262 (127.1.244.193:40615), 3ca43aec4ade4626b571270e98d29dae (127.1.244.195:38743)
I20260807 04:27:06.581705 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.575802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:43053 (local address 127.1.244.193:40615)
0807 04:27:06.576012 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.576016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.576421 (+ 405us) server_negotiation.cc:407] Connection header received
0807 04:27:06.576759 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.576765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.576842 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.576916 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:06.577602 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.578294 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.579998 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.580792 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.580988 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.581065 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.581336 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.581462 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:27:06.581813 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.575642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:40615 (local address 127.1.244.194:43053)
0807 04:27:06.576312 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:06.576344 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:06.576545 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:06.577087 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:06.577090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:06.577110 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:06.577441 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.577448 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.578424 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.578431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.579817 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.579831 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.581016 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.581022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.581176 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.581200 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:27:06.581306 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 04:27:06.581449 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":334,"threads_started":1}
I20260807 04:27:06.582118 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.576605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:59005 (local address 127.1.244.195:38743)
0807 04:27:06.576809 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.576816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.577092 (+ 276us) server_negotiation.cc:407] Connection header received
0807 04:27:06.577360 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.577369 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.577423 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.577515 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:06.578033 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.578725 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.580410 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.581180 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.581356 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.581432 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.581750 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.581871 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 04:27:06.582219 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.576396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:38743 (local address 127.1.244.194:59005)
0807 04:27:06.576980 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:06.576998 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:06.577219 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:06.577529 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:06.577534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:06.577555 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:06.577872 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.577880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.578862 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.578869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.580261 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:06.580273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.581401 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.581416 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:06.581582 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.581606 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 04:27:06.581711 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 04:27:06.581846 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"spinlock_wait_cycles":22784,"thread_start_us":311,"threads_started":1}
I20260807 04:27:06.582780 19916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4938814b793744afa2212d12b63abca1" candidate_uuid: "0ac57887aaaa44b2877127c5e2afc516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0adfc8951f41669f4382a193e4f262" is_pre_election: true
I20260807 04:27:06.583375 19916 raft_consensus.cc:2454] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac57887aaaa44b2877127c5e2afc516 in term 0.
I20260807 04:27:06.583357 20196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4938814b793744afa2212d12b63abca1" candidate_uuid: "0ac57887aaaa44b2877127c5e2afc516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca43aec4ade4626b571270e98d29dae" is_pre_election: true
I20260807 04:27:06.583853 20196 raft_consensus.cc:2454] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ac57887aaaa44b2877127c5e2afc516 in term 0.
I20260807 04:27:06.584079 19987 leader_election.cc:302] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac57887aaaa44b2877127c5e2afc516, 7c0adfc8951f41669f4382a193e4f262; no voters:
I20260807 04:27:06.584471 20268 raft_consensus.cc:2781] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:27:06.584617 20268 raft_consensus.cc:482] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:27:06.584746 20268 raft_consensus.cc:3037] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:06.586923 20268 raft_consensus.cc:504] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.587477 20268 leader_election.cc:288] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0adfc8951f41669f4382a193e4f262 (127.1.244.193:40615), 3ca43aec4ade4626b571270e98d29dae (127.1.244.195:38743)
I20260807 04:27:06.587941 19916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4938814b793744afa2212d12b63abca1" candidate_uuid: "0ac57887aaaa44b2877127c5e2afc516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0adfc8951f41669f4382a193e4f262"
I20260807 04:27:06.588091 20196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4938814b793744afa2212d12b63abca1" candidate_uuid: "0ac57887aaaa44b2877127c5e2afc516" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca43aec4ade4626b571270e98d29dae"
I20260807 04:27:06.588135 19916 raft_consensus.cc:3037] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:06.588295 20196 raft_consensus.cc:3037] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:06.590306 19916 raft_consensus.cc:2454] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac57887aaaa44b2877127c5e2afc516 in term 1.
I20260807 04:27:06.590529 20196 raft_consensus.cc:2454] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ac57887aaaa44b2877127c5e2afc516 in term 1.
I20260807 04:27:06.590685 19987 leader_election.cc:302] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac57887aaaa44b2877127c5e2afc516, 7c0adfc8951f41669f4382a193e4f262; no voters:
I20260807 04:27:06.591020 20268 raft_consensus.cc:2781] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:06.591464 20268 raft_consensus.cc:686] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 1 LEADER]: Becoming Leader. State: Replica: 0ac57887aaaa44b2877127c5e2afc516, State: Running, Role: LEADER
I20260807 04:27:06.591965 20268 consensus_queue.cc:237] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } }
I20260807 04:27:06.596451 19762 catalog_manager.cc:5495] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ac57887aaaa44b2877127c5e2afc516 (127.1.244.194). New cstate: current_term: 1 leader_uuid: "0ac57887aaaa44b2877127c5e2afc516" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac57887aaaa44b2877127c5e2afc516" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 43643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 } health_report { overall_health: UNKNOWN } } }
I20260807 04:27:06.629981 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.624315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.1.244.194:43643)
0807 04:27:06.624560 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.624568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.624697 (+ 129us) server_negotiation.cc:407] Connection header received
0807 04:27:06.624962 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.624966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.625044 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.625199 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:27:06.625621 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.626345 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.627787 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.628227 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.628362 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.628858 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:27:06.629324 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:27:06.629481 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.629615 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.629698 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:27:06.637917 19916 raft_consensus.cc:1261] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 1 FOLLOWER]: Refusing update from remote peer 0ac57887aaaa44b2877127c5e2afc516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:06.638065 20196 raft_consensus.cc:1261] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 1 FOLLOWER]: Refusing update from remote peer 0ac57887aaaa44b2877127c5e2afc516: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:06.639153 20273 consensus_queue.cc:1035] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0adfc8951f41669f4382a193e4f262" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 40615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:06.639492 20268 consensus_queue.cc:1035] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ca43aec4ade4626b571270e98d29dae" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 38743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 04:27:06.641316 19964 tablet.cc:2369] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:06.651369 20276 mvcc.cc:205] Tried to move back new op lower bound from 7315770681898864640 to 7315770681724608512. Current Snapshot: MvccSnapshot[applied={T|T < 7315770681898864640}]
I20260807 04:27:06.673209 19762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:06.673488 19762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:06.676928 19762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2c8036c238449438ea30315daa3f34f: Sending DeleteTablet for 3 replicas of tablet 4938814b793744afa2212d12b63abca1
I20260807 04:27:06.678047 20035 tablet_service.cc:1514] Processing DeleteTablet for tablet 4938814b793744afa2212d12b63abca1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:06 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:52340
I20260807 04:27:06.678180 19895 tablet_service.cc:1514] Processing DeleteTablet for tablet 4938814b793744afa2212d12b63abca1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:06 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:57538
I20260807 04:27:06.679136 20173 tablet_service.cc:1514] Processing DeleteTablet for tablet 4938814b793744afa2212d12b63abca1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:06 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:54414
I20260807 04:27:06.679674 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 19828
I20260807 04:27:06.688817 20287 tablet_replica.cc:330] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: stopping tablet replica
I20260807 04:27:06.689353 20287 raft_consensus.cc:2229] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:06.689953 20287 raft_consensus.cc:2258] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:06.694883 20288 tablet_replica.cc:330] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: stopping tablet replica
I20260807 04:27:06.695436 20286 tablet_replica.cc:330] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: stopping tablet replica
I20260807 04:27:06.695415 20288 raft_consensus.cc:2229] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:06.695840 20286 raft_consensus.cc:2229] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:27:06.696019 20288 raft_consensus.cc:2258] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:06.696583 20286 raft_consensus.cc:2258] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:06.698320 20287 ts_tablet_manager.cc:1905] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:06.698310 20288 ts_tablet_manager.cc:1905] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:06.698484 20286 ts_tablet_manager.cc:1905] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:06.705587 20288 ts_tablet_manager.cc:1918] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:06.705829 20288 log.cc:1196] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/4938814b793744afa2212d12b63abca1
I20260807 04:27:06.706144 20286 ts_tablet_manager.cc:1918] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:06.706316 20286 log.cc:1196] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/4938814b793744afa2212d12b63abca1
I20260807 04:27:06.706641 20288 ts_tablet_manager.cc:1939] T 4938814b793744afa2212d12b63abca1 P 3ca43aec4ade4626b571270e98d29dae: Deleting consensus metadata
I20260807 04:27:06.706882 20286 ts_tablet_manager.cc:1939] T 4938814b793744afa2212d12b63abca1 P 0ac57887aaaa44b2877127c5e2afc516: Deleting consensus metadata
I20260807 04:27:06.708873 19747 catalog_manager.cc:4841] TS 0ac57887aaaa44b2877127c5e2afc516 (127.1.244.194:43643): tablet 4938814b793744afa2212d12b63abca1 (table test-table [id=6ea21cb77cbf453aa4546392f3ab3193]) successfully deleted
I20260807 04:27:06.709241 19749 catalog_manager.cc:4841] TS 3ca43aec4ade4626b571270e98d29dae (127.1.244.195:38743): tablet 4938814b793744afa2212d12b63abca1 (table test-table [id=6ea21cb77cbf453aa4546392f3ab3193]) successfully deleted
I20260807 04:27:06.710609 20287 ts_tablet_manager.cc:1918] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 07 04:27:06 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:27:06.710803 20287 log.cc:1196] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/4938814b793744afa2212d12b63abca1
I20260807 04:27:06.711279 20287 ts_tablet_manager.cc:1939] T 4938814b793744afa2212d12b63abca1 P 7c0adfc8951f41669f4382a193e4f262: Deleting consensus metadata
I20260807 04:27:06.713171 19749 catalog_manager.cc:4841] TS 7c0adfc8951f41669f4382a193e4f262 (127.1.244.193:40615): tablet 4938814b793744afa2212d12b63abca1 (table test-table [id=6ea21cb77cbf453aa4546392f3ab3193]) successfully deleted
I20260807 04:27:06.718456 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.682793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.1.244.193:40615)
0807 04:27:06.682967 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.682973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.694991 (+ 12018us) server_negotiation.cc:407] Connection header received
0807 04:27:06.695469 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.695476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.695562 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.695641 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:06.697644 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.698811 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.700714 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.701341 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.701464 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.712045 (+ 10581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:06.712057 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:06.712075 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:06.712135 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:06.715477 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.716393 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.716400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.716402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.716545 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.716973 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.716976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.716978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.717389 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:06.717618 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.717999 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.718157 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:06.719100 19957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:06.809001 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 19828
I20260807 04:27:06.829852 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 19969
Aug 07 04:27:06 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:27:06.846194 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.830862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.1.244.194:43643)
0807 04:27:06.831063 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.831069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.831093 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:27:06.831197 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.831203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.831276 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.831392 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:06.832727 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.833544 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.834868 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.835321 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.835504 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.840190 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:06.840202 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:06.840218 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:06.840254 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:06.843161 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.843846 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.843854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.843856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.844181 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.844598 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.844604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.844607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.844975 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:06.845321 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.845592 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.845813 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:27:06.847014 20097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:06.957849 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 19969
I20260807 04:27:06.979310 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20108
Aug 07 04:27:06 dist-test-slave-zkpd krb5kdc[19715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:27:06.993908 20258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:06.980083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.1.244.195:38743)
0807 04:27:06.980295 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:06.980299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:06.980317 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:27:06.980483 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:06.980487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:06.980585 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:06.980668 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:06.981877 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.982597 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.984083 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:06.984471 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:06.984570 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:06.988787 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:06.988799 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:06.988813 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:06.988853 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:06.991521 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.992196 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.992215 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.992217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.992377 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:06.992684 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:06.992687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:06.992689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:06.993043 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:06.993258 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:06.993495 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 04:27:06.993649 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 04:27:06.994518 20237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:07.082306 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20108
I20260807 04:27:07.100857 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 19730
I20260807 04:27:07.102057 19795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:07.212908 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 19730
2026-08-07T04:27:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4325 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T04:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:07Z Disabled control of system clock
I20260807 04:27:07.302495 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39859
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:39859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:27:07.644346 20298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:07.644692 20298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:07.644886 20298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:07.652351 20298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:07.652484 20298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:07.652581 20298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:07.652666 20298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:07.652737 20298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:07.661734 20298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:39859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:39859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:07.663022 20298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:07.664943 20298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:07.676609 20303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:07.676594 20304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:07.676748 20306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:07.677413 20298 server_base.cc:1034] running on GCE node
I20260807 04:27:07.678148 20298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:07.679463 20298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:07.680694 20298 hybrid_clock.cc:648] HybridClock initialized: now 1786076827680655 us; error 65 us; skew 500 ppm
I20260807 04:27:07.681036 20298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:07.683256 20298 webserver.cc:460] Webserver started at http://127.1.244.254:34167/ using document root <none> and password file <none>
I20260807 04:27:07.683956 20298 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:07.684119 20298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:07.684435 20298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:07.687207 20298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "c1243ef164f5431dbb7a7faf215da2c0"
format_stamp: "Formatted at 2026-08-07 04:27:07 on dist-test-slave-zkpd"
server_key: "9deb4740a3b3d313fa10b6931e00602e"
server_key_iv: "8dca471e936c7fdab5e20016f9b2761e"
server_key_version: "encryptionkey@0"
I20260807 04:27:07.688058 20298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "c1243ef164f5431dbb7a7faf215da2c0"
format_stamp: "Formatted at 2026-08-07 04:27:07 on dist-test-slave-zkpd"
server_key: "9deb4740a3b3d313fa10b6931e00602e"
server_key_iv: "8dca471e936c7fdab5e20016f9b2761e"
server_key_version: "encryptionkey@0"
I20260807 04:27:07.692919 20298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 04:27:07.696334 20312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:07.697585 20298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:27:07.697769 20298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "c1243ef164f5431dbb7a7faf215da2c0"
format_stamp: "Formatted at 2026-08-07 04:27:07 on dist-test-slave-zkpd"
server_key: "9deb4740a3b3d313fa10b6931e00602e"
server_key_iv: "8dca471e936c7fdab5e20016f9b2761e"
server_key_version: "encryptionkey@0"
I20260807 04:27:07.697988 20298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:07.718593 20298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:07.719391 20298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:07.719681 20298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:07.738276 20298 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:39859
I20260807 04:27:07.738296 20367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:39859 every 8 connection(s)
I20260807 04:27:07.739576 20298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:07.743153 20368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:07.747614 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20298
I20260807 04:27:07.747816 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:07.748283 2003 external_mini_cluster.cc:1428] Setting key b7c16d6a8999f939d03a9cb9342a4a04
I20260807 04:27:07.755484 20368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0: Bootstrap starting.
I20260807 04:27:07.758952 20368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:07.760311 20368 log.cc:824] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:07.763521 20368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0: No bootstrap required, opened a new log
I20260807 04:27:07.763787 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:07.750196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.1.244.254:39859)
0807 04:27:07.751298 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:07.751320 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:07.751381 (+ 61us) server_negotiation.cc:407] Connection header received
0807 04:27:07.753922 (+ 2541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:07.753951 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:07.754492 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:07.754900 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:07.755424 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:07.757384 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:07.758654 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:07.759798 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:07.760028 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:07.761127 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:07.761134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:07.761146 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:07.761790 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:07.761883 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:07.761918 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:07.762103 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:07.762299 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 04:27:07.762596 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":344,"threads_started":1}
I20260807 04:27:07.770872 20368 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1243ef164f5431dbb7a7faf215da2c0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39859 } }
I20260807 04:27:07.771180 20368 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:07.771252 20368 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1243ef164f5431dbb7a7faf215da2c0, State: Initialized, Role: FOLLOWER
I20260807 04:27:07.771750 20368 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1243ef164f5431dbb7a7faf215da2c0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39859 } }
I20260807 04:27:07.772092 20368 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:07.772266 20368 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:07.772472 20368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:07.774655 20368 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1243ef164f5431dbb7a7faf215da2c0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39859 } }
I20260807 04:27:07.775126 20368 leader_election.cc:302] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1243ef164f5431dbb7a7faf215da2c0; no voters:
I20260807 04:27:07.775676 20368 leader_election.cc:288] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:07.775815 20373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:07.776629 20373 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [term 1 LEADER]: Becoming Leader. State: Replica: c1243ef164f5431dbb7a7faf215da2c0, State: Running, Role: LEADER
I20260807 04:27:07.777170 20373 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1243ef164f5431dbb7a7faf215da2c0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39859 } }
I20260807 04:27:07.777733 20368 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:07.780426 20374 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1243ef164f5431dbb7a7faf215da2c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1243ef164f5431dbb7a7faf215da2c0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39859 } } }
I20260807 04:27:07.780838 20374 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:07.780480 20375 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1243ef164f5431dbb7a7faf215da2c0. Latest consensus state: current_term: 1 leader_uuid: "c1243ef164f5431dbb7a7faf215da2c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1243ef164f5431dbb7a7faf215da2c0" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 39859 } } }
I20260807 04:27:07.781222 20375 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:07.782914 20380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:07.787539 20380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:07.796053 20380 catalog_manager.cc:1347] Generated new cluster ID: f8823cab86524bafbc63b0a10c0c8efa
I20260807 04:27:07.796161 20380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:07.822929 20380 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:07.824299 20380 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:07.830978 20380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0: Generated new TSK 0
I20260807 04:27:07.831722 20380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:07.843822 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39859
--builtin_ntp_servers=127.1.244.212:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:27:08.176854 20396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:08.177122 20396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:08.177207 20396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:08.184448 20396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:08.184535 20396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:08.184726 20396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:27:08.193590 20396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:08.194839 20396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:08.197266 20396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:08.210235 20404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:08.210350 20401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:08.210320 20402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:08.211253 20396 server_base.cc:1034] running on GCE node
I20260807 04:27:08.211807 20396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:08.212731 20396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:08.213954 20396 hybrid_clock.cc:648] HybridClock initialized: now 1786076828213895 us; error 46 us; skew 500 ppm
I20260807 04:27:08.214253 20396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:08.220579 20396 webserver.cc:460] Webserver started at http://127.1.244.193:40117/ using document root <none> and password file <none>
I20260807 04:27:08.221222 20396 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:08.221357 20396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:08.221678 20396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:08.224514 20396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f"
format_stamp: "Formatted at 2026-08-07 04:27:08 on dist-test-slave-zkpd"
server_key: "1bd4ebcef6acf00ea6376cb39e63c55a"
server_key_iv: "e3d92509e4c5859b60c964197c7f151f"
server_key_version: "encryptionkey@0"
I20260807 04:27:08.225278 20396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f"
format_stamp: "Formatted at 2026-08-07 04:27:08 on dist-test-slave-zkpd"
server_key: "1bd4ebcef6acf00ea6376cb39e63c55a"
server_key_iv: "e3d92509e4c5859b60c964197c7f151f"
server_key_version: "encryptionkey@0"
I20260807 04:27:08.230170 20396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:08.233587 20410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:08.234834 20396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:27:08.235011 20396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f"
format_stamp: "Formatted at 2026-08-07 04:27:08 on dist-test-slave-zkpd"
server_key: "1bd4ebcef6acf00ea6376cb39e63c55a"
server_key_iv: "e3d92509e4c5859b60c964197c7f151f"
server_key_version: "encryptionkey@0"
I20260807 04:27:08.235227 20396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:08.257313 20396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:08.258080 20396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:08.258399 20396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:08.259318 20396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:08.260782 20396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:08.260902 20396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:08.261032 20396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:08.261122 20396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:08.307340 20396 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:36705
I20260807 04:27:08.307427 20527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:36705 every 8 connection(s)
I20260807 04:27:08.309302 20396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:27:08.314556 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20396
I20260807 04:27:08.314760 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:27:08.315227 2003 external_mini_cluster.cc:1428] Setting key 31fec1e4dc86da248c1d4699b449ef70
I20260807 04:27:08.318166 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39859
--builtin_ntp_servers=127.1.244.212:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:27:08.325239 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:08.312313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:54889 (local address 127.1.244.254:39859)
0807 04:27:08.312857 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:08.312865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:08.314347 (+ 1482us) server_negotiation.cc:407] Connection header received
0807 04:27:08.317447 (+ 3100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:08.317452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:08.317538 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:08.317658 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:08.319198 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.320454 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.322003 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.322726 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.322869 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:08.324260 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:08.324268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:08.324276 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:08.324302 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:08.324359 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:08.324377 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:08.324473 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:08.324671 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 04:27:08.324781 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":250,"threads_started":1}
I20260807 04:27:08.326016 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:08.312904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39859 (local address 127.1.244.193:54889)
0807 04:27:08.314145 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:08.314196 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:08.317115 (+ 2919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:08.317953 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:08.317967 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:08.318096 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:08.318987 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:08.319014 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.320626 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.320630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:08.321857 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:08.321869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.322927 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.322933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:08.323137 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:08.323860 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:08.323863 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:08.323959 (+ 96us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:08.324060 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:08.324476 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:08.324484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:08.324490 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:27:08.324628 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 04:27:08.324922 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":337,"threads_started":1}
I20260807 04:27:08.327618 20528 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39859
I20260807 04:27:08.327986 20528 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:08.328680 20528 heartbeater.cc:511] Master 127.1.244.254:39859 requested a full tablet report, sending...
I20260807 04:27:08.330899 20330 ts_manager.cc:194] Registered new tserver with Master: 1fabf57025ef4fd7a7c3f5eb3d38bc0f (127.1.244.193:36705)
I20260807 04:27:08.332476 20330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:54889
W20260807 04:27:08.649015 20533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:08.649276 20533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:08.649428 20533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:08.657078 20533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:08.657173 20533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:08.657357 20533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:27:08.665926 20533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:08.667078 20533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:08.668843 20533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:08.681833 20542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:08.681877 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:08.681946 20540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:08.683171 20533 server_base.cc:1034] running on GCE node
I20260807 04:27:08.683730 20533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:08.684633 20533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:08.685818 20533 hybrid_clock.cc:648] HybridClock initialized: now 1786076828685753 us; error 55 us; skew 500 ppm
I20260807 04:27:08.686126 20533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:08.688282 20533 webserver.cc:460] Webserver started at http://127.1.244.194:46027/ using document root <none> and password file <none>
I20260807 04:27:08.688912 20533 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:08.689023 20533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:08.689340 20533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:08.692023 20533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "3781bc2674164615b3196a11308efe29"
format_stamp: "Formatted at 2026-08-07 04:27:08 on dist-test-slave-zkpd"
server_key: "6029b7811fa28d694a8487ebdfbe9ec6"
server_key_iv: "717b5703815b708221d682c5613fb30e"
server_key_version: "encryptionkey@0"
I20260807 04:27:08.692790 20533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "3781bc2674164615b3196a11308efe29"
format_stamp: "Formatted at 2026-08-07 04:27:08 on dist-test-slave-zkpd"
server_key: "6029b7811fa28d694a8487ebdfbe9ec6"
server_key_iv: "717b5703815b708221d682c5613fb30e"
server_key_version: "encryptionkey@0"
I20260807 04:27:08.697642 20533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 04:27:08.701061 20548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:08.702587 20533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 04:27:08.702755 20533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "3781bc2674164615b3196a11308efe29"
format_stamp: "Formatted at 2026-08-07 04:27:08 on dist-test-slave-zkpd"
server_key: "6029b7811fa28d694a8487ebdfbe9ec6"
server_key_iv: "717b5703815b708221d682c5613fb30e"
server_key_version: "encryptionkey@0"
I20260807 04:27:08.702975 20533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:08.720677 20533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:08.721459 20533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:08.721768 20533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:08.722740 20533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:08.724316 20533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:08.724447 20533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:08.724586 20533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:08.724674 20533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:08.759069 20533 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41351
I20260807 04:27:08.759119 20666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41351 every 8 connection(s)
I20260807 04:27:08.760378 20533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:27:08.768850 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20533
I20260807 04:27:08.769044 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:27:08.769513 2003 external_mini_cluster.cc:1428] Setting key 4a039dab3588a74360aeadc1f594b4ec
I20260807 04:27:08.772358 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39859
--builtin_ntp_servers=127.1.244.212:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:27:08.772554 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:08.762851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:58895 (local address 127.1.244.254:39859)
0807 04:27:08.763034 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:08.763039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:08.764357 (+ 1318us) server_negotiation.cc:407] Connection header received
0807 04:27:08.766069 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:08.766075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:08.766157 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:08.766264 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:08.767253 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.768007 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.769740 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.770283 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.770445 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:08.771759 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:08.771764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:08.771767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:08.771784 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:08.771816 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:08.771827 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:08.771953 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:08.772163 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 04:27:08.772262 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:27:08.773524 20669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:08.763327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39859 (local address 127.1.244.194:58895)
0807 04:27:08.764211 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:08.764245 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:08.765799 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:08.766484 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:08.766492 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:08.766569 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:08.767096 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:08.767112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.768160 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.768165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:08.769523 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:08.769536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:08.770521 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:08.770528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:08.770753 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:08.771445 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:08.771447 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:08.771483 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:08.771557 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:08.771944 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:08.771956 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:08.771967 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 04:27:08.772154 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 04:27:08.772571 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":262,"threads_started":1}
I20260807 04:27:08.774921 20667 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39859
I20260807 04:27:08.775234 20667 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:08.776053 20667 heartbeater.cc:511] Master 127.1.244.254:39859 requested a full tablet report, sending...
I20260807 04:27:08.777467 20330 ts_manager.cc:194] Registered new tserver with Master: 3781bc2674164615b3196a11308efe29 (127.1.244.194:41351)
I20260807 04:27:08.778724 20330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:58895
W20260807 04:27:09.105041 20670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:09.105309 20670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:09.105396 20670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:09.112515 20670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:09.112612 20670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:09.112715 20670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:27:09.121582 20670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:39859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:09.122766 20670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:09.124478 20670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:09.137383 20677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:09.137454 20680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:09.137485 20678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:09.138458 20670 server_base.cc:1034] running on GCE node
I20260807 04:27:09.139065 20670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:09.140112 20670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:09.141314 20670 hybrid_clock.cc:648] HybridClock initialized: now 1786076829141264 us; error 37 us; skew 500 ppm
I20260807 04:27:09.141625 20670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:09.144023 20670 webserver.cc:460] Webserver started at http://127.1.244.195:39613/ using document root <none> and password file <none>
I20260807 04:27:09.144729 20670 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:09.144850 20670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:09.145171 20670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:09.147962 20670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "f5c63ab5844746c2baf330c0dc44d91c"
format_stamp: "Formatted at 2026-08-07 04:27:09 on dist-test-slave-zkpd"
server_key: "a7fc94a05d16ff00b94db4774f5f45c2"
server_key_iv: "4cc54a300b3f990a891cc29fd3f21b7d"
server_key_version: "encryptionkey@0"
I20260807 04:27:09.148772 20670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "f5c63ab5844746c2baf330c0dc44d91c"
format_stamp: "Formatted at 2026-08-07 04:27:09 on dist-test-slave-zkpd"
server_key: "a7fc94a05d16ff00b94db4774f5f45c2"
server_key_iv: "4cc54a300b3f990a891cc29fd3f21b7d"
server_key_version: "encryptionkey@0"
I20260807 04:27:09.153652 20670 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 04:27:09.156970 20687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:09.158118 20670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 04:27:09.158289 20670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "f5c63ab5844746c2baf330c0dc44d91c"
format_stamp: "Formatted at 2026-08-07 04:27:09 on dist-test-slave-zkpd"
server_key: "a7fc94a05d16ff00b94db4774f5f45c2"
server_key_iv: "4cc54a300b3f990a891cc29fd3f21b7d"
server_key_version: "encryptionkey@0"
I20260807 04:27:09.158500 20670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:09.179499 20670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:09.180269 20670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:09.180598 20670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:09.181524 20670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:09.182898 20670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:09.183013 20670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:09.183145 20670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:09.183243 20670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:09.217442 20670 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:40587
I20260807 04:27:09.217530 20803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:40587 every 8 connection(s)
I20260807 04:27:09.218689 20670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:27:09.221210 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20670
I20260807 04:27:09.221390 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:27:09.221858 2003 external_mini_cluster.cc:1428] Setting key 8dd6be8a773cd52a93679e5d65756fe8
I20260807 04:27:09.231133 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.221166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:37985 (local address 127.1.244.254:39859)
0807 04:27:09.221393 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.221399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.222915 (+ 1516us) server_negotiation.cc:407] Connection header received
0807 04:27:09.225110 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.225114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.225179 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.225264 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:09.226463 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.227182 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.228817 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.229223 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.229315 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.230489 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:09.230494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:09.230497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:09.230513 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:09.230545 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:09.230556 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:09.230623 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.230822 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.230895 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 04:27:09.232048 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.221683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:39859 (local address 127.1.244.195:37985)
0807 04:27:09.222755 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.222799 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.224877 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.225455 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.225462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.225545 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:09.226315 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.226333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.227330 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.227334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.228699 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.228712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.229402 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.229407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.229590 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.230200 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:09.230203 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:09.230237 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:09.230311 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:09.230635 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:09.230642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:09.230648 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.230784 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.231046 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":381,"threads_started":1}
I20260807 04:27:09.233266 20804 heartbeater.cc:347] Connected to a master server at 127.1.244.254:39859
I20260807 04:27:09.233536 20804 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:09.234194 20804 heartbeater.cc:511] Master 127.1.244.254:39859 requested a full tablet report, sending...
I20260807 04:27:09.235522 20330 ts_manager.cc:194] Registered new tserver with Master: f5c63ab5844746c2baf330c0dc44d91c (127.1.244.195:40587)
I20260807 04:27:09.236363 20330 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:37985
I20260807 04:27:09.237000 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:27:09.246515 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.241246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.1.244.254:39859)
0807 04:27:09.241485 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.241494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.241832 (+ 338us) server_negotiation.cc:407] Connection header received
0807 04:27:09.241960 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.241964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.242032 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.242110 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:09.242528 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.243173 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.244494 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.244866 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.244965 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.245886 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:09.245891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:09.245894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:09.245917 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:09.245945 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:09.245957 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:09.246023 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.246140 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.246203 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:27:09.249758 20330 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:27:09.256573 20330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:27:09.292956 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.268045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41351 (local address 127.0.0.1:33670)
0807 04:27:09.282719 (+ 14674us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.282740 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.282886 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.285978 (+ 3092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.285983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.286016 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.286393 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.286409 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.288354 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.288361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.290051 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.290063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.291991 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.291997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.292193 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.292246 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.292375 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.292569 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14500,"mutex_wait_us":59,"thread_start_us":10731,"threads_started":1}
I20260807 04:27:09.293344 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.267944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.1.244.193:36705)
0807 04:27:09.284949 (+ 17005us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.284961 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.284997 (+ 36us) server_negotiation.cc:407] Connection header received
0807 04:27:09.285095 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.285102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.285316 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.285510 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.286307 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.287981 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.290762 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.292214 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.292357 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.292575 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.292800 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.292999 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16799,"thread_start_us":16667,"threads_started":1}
I20260807 04:27:09.293437 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.267758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53348)
0807 04:27:09.279170 (+ 11412us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.279207 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.279372 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.285516 (+ 6144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.285519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.285558 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.286084 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.286103 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.288116 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.288121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.290553 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.290568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.292425 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.292435 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.292612 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.292645 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.292748 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.292873 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11166,"mutex_wait_us":100,"thread_start_us":15209,"threads_started":1}
I20260807 04:27:09.293916 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.269391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40587 (local address 127.0.0.1:55302)
0807 04:27:09.283574 (+ 14183us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.283594 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.283711 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.285222 (+ 1511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.285229 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.285259 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.285714 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.285733 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.287713 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.287718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.290589 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.290601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.292919 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.292925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.293220 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.293256 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.293371 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.293531 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14035,"mutex_wait_us":219,"thread_start_us":13864,"threads_started":1}
I20260807 04:27:09.294415 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.268578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33670 (local address 127.1.244.194:41351)
0807 04:27:09.285289 (+ 16711us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.285300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.285341 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:27:09.285448 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.285456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.285684 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.286581 (+ 897us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.286780 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.288203 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.290229 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.291470 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.291596 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.291838 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.294004 (+ 2166us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.294172 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16506,"thread_start_us":16406,"threads_started":1}
I20260807 04:27:09.294696 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.269760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.1.244.195:40587)
0807 04:27:09.284368 (+ 14608us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.284379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.284424 (+ 45us) server_negotiation.cc:407] Connection header received
0807 04:27:09.284549 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.284561 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.284819 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.285074 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.286117 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.287531 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.290931 (+ 3400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.292341 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.292478 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.292784 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.294145 (+ 1361us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.294308 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14415,"thread_start_us":14250,"threads_started":1}
I20260807 04:27:09.297444 20459 tablet_service.cc:1467] Processing CreateTablet for tablet b017f51f20ff4b3c9a46bb612160801b (DEFAULT_TABLE table=test-table [id=e2b5653e5b60482889647d994fb076b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:09.298645 20459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017f51f20ff4b3c9a46bb612160801b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:09.299324 20736 tablet_service.cc:1467] Processing CreateTablet for tablet b017f51f20ff4b3c9a46bb612160801b (DEFAULT_TABLE table=test-table [id=e2b5653e5b60482889647d994fb076b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:09.299324 20598 tablet_service.cc:1467] Processing CreateTablet for tablet b017f51f20ff4b3c9a46bb612160801b (DEFAULT_TABLE table=test-table [id=e2b5653e5b60482889647d994fb076b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:09.300691 20736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017f51f20ff4b3c9a46bb612160801b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:09.300686 20598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b017f51f20ff4b3c9a46bb612160801b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:09.318006 20824 tablet_bootstrap.cc:492] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Bootstrap starting.
I20260807 04:27:09.317179 20823 tablet_bootstrap.cc:492] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Bootstrap starting.
I20260807 04:27:09.321165 20825 tablet_bootstrap.cc:492] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Bootstrap starting.
I20260807 04:27:09.321681 20824 tablet_bootstrap.cc:654] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:09.322575 20823 tablet_bootstrap.cc:654] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:09.323043 20824 log.cc:824] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:09.323673 20823 log.cc:824] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:09.325860 20825 tablet_bootstrap.cc:654] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:09.326339 20824 tablet_bootstrap.cc:492] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: No bootstrap required, opened a new log
I20260807 04:27:09.326606 20824 ts_tablet_manager.cc:1397] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 04:27:09.326726 20823 tablet_bootstrap.cc:492] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: No bootstrap required, opened a new log
I20260807 04:27:09.326915 20825 log.cc:824] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:09.326990 20823 ts_tablet_manager.cc:1397] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 04:27:09.329722 20825 tablet_bootstrap.cc:492] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: No bootstrap required, opened a new log
I20260807 04:27:09.329969 20825 ts_tablet_manager.cc:1397] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 04:27:09.335398 20528 heartbeater.cc:503] Master 127.1.244.254:39859 was elected leader, sending a full tablet report...
I20260807 04:27:09.337391 20824 raft_consensus.cc:348] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.337843 20824 raft_consensus.cc:374] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:09.337950 20824 raft_consensus.cc:729] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c63ab5844746c2baf330c0dc44d91c, State: Initialized, Role: FOLLOWER
I20260807 04:27:09.337720 20825 raft_consensus.cc:348] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.337757 20823 raft_consensus.cc:348] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.338115 20825 raft_consensus.cc:374] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:09.338207 20825 raft_consensus.cc:729] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3781bc2674164615b3196a11308efe29, State: Initialized, Role: FOLLOWER
I20260807 04:27:09.338222 20823 raft_consensus.cc:374] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:09.338373 20823 raft_consensus.cc:729] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fabf57025ef4fd7a7c3f5eb3d38bc0f, State: Initialized, Role: FOLLOWER
I20260807 04:27:09.338693 20824 consensus_queue.cc:260] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.338873 20825 consensus_queue.cc:260] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.339103 20823 consensus_queue.cc:260] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.342549 20667 heartbeater.cc:503] Master 127.1.244.254:39859 was elected leader, sending a full tablet report...
I20260807 04:27:09.343080 20804 heartbeater.cc:503] Master 127.1.244.254:39859 was elected leader, sending a full tablet report...
I20260807 04:27:09.344218 20823 ts_tablet_manager.cc:1428] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Time spent starting tablet: real 0.017s user 0.017s sys 0.001s
I20260807 04:27:09.345155 20824 ts_tablet_manager.cc:1428] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Time spent starting tablet: real 0.018s user 0.009s sys 0.010s
I20260807 04:27:09.345857 20825 ts_tablet_manager.cc:1428] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Time spent starting tablet: real 0.016s user 0.005s sys 0.009s
W20260807 04:27:09.470419 20805 tablet.cc:2369] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:09.479542 20831 raft_consensus.cc:482] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:09.479966 20831 raft_consensus.cc:504] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.481870 20831 leader_election.cc:288] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3781bc2674164615b3196a11308efe29 (127.1.244.194:41351), f5c63ab5844746c2baf330c0dc44d91c (127.1.244.195:40587)
I20260807 04:27:09.488538 20830 raft_consensus.cc:482] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:09.489104 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.482403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:42763 (local address 127.1.244.195:40587)
0807 04:27:09.482727 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.482733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.483362 (+ 629us) server_negotiation.cc:407] Connection header received
0807 04:27:09.483568 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.483574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.483651 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.483746 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.484460 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.485309 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.487358 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.488125 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.488295 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.488367 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.488713 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.488855 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260807 04:27:09.489163 20833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.482154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40587 (local address 127.1.244.193:42763)
0807 04:27:09.483258 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.483291 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.483422 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.483777 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.483780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.483813 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.484298 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.484311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.485453 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.485457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.487127 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.487140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.488299 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.488306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.488497 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.488539 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.488661 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.488792 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":115,"thread_start_us":743,"threads_started":1}
I20260807 04:27:09.488886 20830 raft_consensus.cc:504] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.489228 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.482274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:34129 (local address 127.1.244.194:41351)
0807 04:27:09.482533 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.482543 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.483174 (+ 631us) server_negotiation.cc:407] Connection header received
0807 04:27:09.483367 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.483375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.483491 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.483582 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.484265 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.485043 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.487314 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.488167 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.488315 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.488362 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.488833 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.488958 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 04:27:09.489864 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.482058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41351 (local address 127.1.244.193:34129)
0807 04:27:09.483042 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.483065 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.483222 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.483617 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.483622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.483647 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.484104 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.484117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.485165 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.485170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.487122 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.487134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.488476 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.488481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.488600 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.488673 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.489385 (+ 712us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.489537 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":78,"thread_start_us":579,"threads_started":1}
I20260807 04:27:09.490134 20756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017f51f20ff4b3c9a46bb612160801b" candidate_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c63ab5844746c2baf330c0dc44d91c" is_pre_election: true
I20260807 04:27:09.490597 20756 raft_consensus.cc:2454] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fabf57025ef4fd7a7c3f5eb3d38bc0f in term 0.
I20260807 04:27:09.490732 20830 leader_election.cc:288] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fabf57025ef4fd7a7c3f5eb3d38bc0f (127.1.244.193:36705), f5c63ab5844746c2baf330c0dc44d91c (127.1.244.195:40587)
I20260807 04:27:09.491382 20413 leader_election.cc:302] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fabf57025ef4fd7a7c3f5eb3d38bc0f, f5c63ab5844746c2baf330c0dc44d91c; no voters:
I20260807 04:27:09.491971 20831 raft_consensus.cc:2781] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:27:09.492126 20831 raft_consensus.cc:482] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:27:09.492224 20831 raft_consensus.cc:3037] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:09.495158 20831 raft_consensus.cc:504] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.495711 20831 leader_election.cc:288] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [CANDIDATE]: Term 1 election: Requested vote from peers 3781bc2674164615b3196a11308efe29 (127.1.244.194:41351), f5c63ab5844746c2baf330c0dc44d91c (127.1.244.195:40587)
I20260807 04:27:09.496188 20756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017f51f20ff4b3c9a46bb612160801b" candidate_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c63ab5844746c2baf330c0dc44d91c"
I20260807 04:27:09.496378 20756 raft_consensus.cc:3037] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:09.496347 20618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017f51f20ff4b3c9a46bb612160801b" candidate_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3781bc2674164615b3196a11308efe29"
I20260807 04:27:09.496562 20619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017f51f20ff4b3c9a46bb612160801b" candidate_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3781bc2674164615b3196a11308efe29" is_pre_election: true
I20260807 04:27:09.496990 20619 raft_consensus.cc:2454] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fabf57025ef4fd7a7c3f5eb3d38bc0f in term 0.
I20260807 04:27:09.499632 20756 raft_consensus.cc:2454] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fabf57025ef4fd7a7c3f5eb3d38bc0f in term 1.
I20260807 04:27:09.499994 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.491401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:60163 (local address 127.1.244.195:40587)
0807 04:27:09.491565 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.491571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.492879 (+ 1308us) server_negotiation.cc:407] Connection header received
0807 04:27:09.492989 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.492996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.493061 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.493149 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.493609 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.494679 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.497504 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.498765 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.498887 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.498948 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.499547 (+ 599us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.499697 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:27:09.500155 20834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.491339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:40587 (local address 127.1.244.194:60163)
0807 04:27:09.492780 (+ 1441us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.492794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.492888 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.493166 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.493169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.493192 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.493460 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.493466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.494799 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.494802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.497268 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.497283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.499130 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.499136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.499315 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.499383 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.499500 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.499653 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"thread_start_us":382,"threads_started":1}
I20260807 04:27:09.500097 20413 leader_election.cc:302] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fabf57025ef4fd7a7c3f5eb3d38bc0f, f5c63ab5844746c2baf330c0dc44d91c; no voters: 3781bc2674164615b3196a11308efe29
I20260807 04:27:09.500530 20831 raft_consensus.cc:2781] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:09.501012 20755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017f51f20ff4b3c9a46bb612160801b" candidate_uuid: "3781bc2674164615b3196a11308efe29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c63ab5844746c2baf330c0dc44d91c" is_pre_election: true
I20260807 04:27:09.501420 20755 raft_consensus.cc:2379] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3781bc2674164615b3196a11308efe29 in current term 1: Already voted for candidate 1fabf57025ef4fd7a7c3f5eb3d38bc0f in this term.
I20260807 04:27:09.501490 20831 raft_consensus.cc:686] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 1 LEADER]: Becoming Leader. State: Replica: 1fabf57025ef4fd7a7c3f5eb3d38bc0f, State: Running, Role: LEADER
I20260807 04:27:09.502066 20831 consensus_queue.cc:237] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } }
I20260807 04:27:09.503528 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.491159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.1.244.194:33445)
0807 04:27:09.492136 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.492156 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:09.492281 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:09.496236 (+ 3955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:09.496241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:09.496265 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:09.498076 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.498087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.498962 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.498969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.501031 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:09.501048 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.502806 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.502819 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:09.502963 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.502997 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 04:27:09.503104 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 04:27:09.503226 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":1343,"threads_started":1}
I20260807 04:27:09.503860 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.491820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:33445 (local address 127.1.244.193:36705)
0807 04:27:09.495977 (+ 4157us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.495982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.495997 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:27:09.496050 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.496056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.496121 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.496218 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:09.498216 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.498830 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.501258 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.502620 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.502804 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.502865 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.503134 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.503358 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4006}
I20260807 04:27:09.504688 20481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b017f51f20ff4b3c9a46bb612160801b" candidate_uuid: "3781bc2674164615b3196a11308efe29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" is_pre_election: true
I20260807 04:27:09.505612 20552 leader_election.cc:302] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3781bc2674164615b3196a11308efe29; no voters: 1fabf57025ef4fd7a7c3f5eb3d38bc0f, f5c63ab5844746c2baf330c0dc44d91c
I20260807 04:27:09.506024 20830 raft_consensus.cc:3037] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:09.507107 20330 catalog_manager.cc:5495] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fabf57025ef4fd7a7c3f5eb3d38bc0f (127.1.244.193). New cstate: current_term: 1 leader_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fabf57025ef4fd7a7c3f5eb3d38bc0f" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 36705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 } health_report { overall_health: UNKNOWN } } }
I20260807 04:27:09.508937 20830 raft_consensus.cc:2726] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 04:27:09.512377 20668 tablet.cc:2369] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:09.537165 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.532232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.1.244.193:36705)
0807 04:27:09.532387 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.532391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.532476 (+ 85us) server_negotiation.cc:407] Connection header received
0807 04:27:09.532642 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.532646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.532751 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.532870 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:27:09.533183 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.533940 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.535295 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.535718 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.535805 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.536261 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:27:09.536718 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:27:09.536831 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.536901 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.536958 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:27:09.544939 20618 raft_consensus.cc:1261] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 1 FOLLOWER]: Refusing update from remote peer 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:09.544996 20755 raft_consensus.cc:1261] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 1 FOLLOWER]: Refusing update from remote peer 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:09.545742 20836 consensus_queue.cc:1035] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3781bc2674164615b3196a11308efe29" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:09.546070 20831 consensus_queue.cc:1035] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c63ab5844746c2baf330c0dc44d91c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 40587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:09.557396 20840 mvcc.cc:205] Tried to move back new op lower bound from 7315770693806632960 to 7315770693645258752. Current Snapshot: MvccSnapshot[applied={T|T < 7315770693806632960}]
I20260807 04:27:09.560101 20839 mvcc.cc:205] Tried to move back new op lower bound from 7315770693806632960 to 7315770693645258752. Current Snapshot: MvccSnapshot[applied={T|T < 7315770693806632960}]
I20260807 04:27:09.574545 20329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:09.574867 20329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:09.578890 20329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1243ef164f5431dbb7a7faf215da2c0: Sending DeleteTablet for 3 replicas of tablet b017f51f20ff4b3c9a46bb612160801b
I20260807 04:27:09.580063 20459 tablet_service.cc:1514] Processing DeleteTablet for tablet b017f51f20ff4b3c9a46bb612160801b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:09 UTC) from {username='slave'} at 127.0.0.1:53348
I20260807 04:27:09.580132 20736 tablet_service.cc:1514] Processing DeleteTablet for tablet b017f51f20ff4b3c9a46bb612160801b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:09 UTC) from {username='slave'} at 127.0.0.1:55302
I20260807 04:27:09.580804 20598 tablet_service.cc:1514] Processing DeleteTablet for tablet b017f51f20ff4b3c9a46bb612160801b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:09 UTC) from {username='slave'} at 127.0.0.1:33670
I20260807 04:27:09.581735 20851 tablet_replica.cc:330] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: stopping tablet replica
I20260807 04:27:09.582144 20851 raft_consensus.cc:2229] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:09.582602 20851 raft_consensus.cc:2258] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:09.582901 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20396
I20260807 04:27:09.584800 20851 ts_tablet_manager.cc:1905] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:09.591769 20851 ts_tablet_manager.cc:1918] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:09.591952 20851 log.cc:1196] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/b017f51f20ff4b3c9a46bb612160801b
I20260807 04:27:09.592509 20851 ts_tablet_manager.cc:1939] T b017f51f20ff4b3c9a46bb612160801b P 3781bc2674164615b3196a11308efe29: Deleting consensus metadata
I20260807 04:27:09.594307 20313 catalog_manager.cc:4841] TS 3781bc2674164615b3196a11308efe29 (127.1.244.194:41351): tablet b017f51f20ff4b3c9a46bb612160801b (table test-table [id=e2b5653e5b60482889647d994fb076b3]) successfully deleted
I20260807 04:27:09.594707 20849 tablet_replica.cc:330] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: stopping tablet replica
I20260807 04:27:09.595278 20849 raft_consensus.cc:2229] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:27:09.596185 20849 raft_consensus.cc:2258] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:09.598912 20849 ts_tablet_manager.cc:1905] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:09.601794 20850 tablet_replica.cc:330] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: stopping tablet replica
I20260807 04:27:09.602236 20850 raft_consensus.cc:2229] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:09.602761 20850 raft_consensus.cc:2258] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:09.604998 20850 ts_tablet_manager.cc:1905] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:09.605378 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.584151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.1.244.193:36705)
0807 04:27:09.584747 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.584756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.595017 (+ 10261us) server_negotiation.cc:407] Connection header received
0807 04:27:09.595305 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.595312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.595450 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.595527 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:09.596265 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.597435 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.599439 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.600160 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.600258 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.603523 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:09.603535 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:09.603552 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:09.604355 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:09.604480 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:09.604511 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:09.604612 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.604870 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.604988 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434}
I20260807 04:27:09.606587 20523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:09.613636 20850 ts_tablet_manager.cc:1918] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:09.613838 20850 log.cc:1196] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/b017f51f20ff4b3c9a46bb612160801b
I20260807 04:27:09.614344 20850 ts_tablet_manager.cc:1939] T b017f51f20ff4b3c9a46bb612160801b P f5c63ab5844746c2baf330c0dc44d91c: Deleting consensus metadata
I20260807 04:27:09.616174 20316 catalog_manager.cc:4841] TS f5c63ab5844746c2baf330c0dc44d91c (127.1.244.195:40587): tablet b017f51f20ff4b3c9a46bb612160801b (table test-table [id=e2b5653e5b60482889647d994fb076b3]) successfully deleted
I20260807 04:27:09.704061 20849 ts_tablet_manager.cc:1918] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:09.704362 20849 log.cc:1196] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/b017f51f20ff4b3c9a46bb612160801b
I20260807 04:27:09.704950 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20396
I20260807 04:27:09.705062 20849 ts_tablet_manager.cc:1939] T b017f51f20ff4b3c9a46bb612160801b P 1fabf57025ef4fd7a7c3f5eb3d38bc0f: Deleting consensus metadata
W20260807 04:27:09.726397 20317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:36705 (error 108)
I20260807 04:27:09.726630 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20533
I20260807 04:27:09.728494 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.728029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53370)
0807 04:27:09.728214 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.728349 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 04:27:09.728776 20317 catalog_manager.cc:4584] TS 1fabf57025ef4fd7a7c3f5eb3d38bc0f (127.1.244.193:36705): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b017f51f20ff4b3c9a46bb612160801b: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
I20260807 04:27:09.742776 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.727423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.1.244.194:41351)
0807 04:27:09.727612 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.727616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.727722 (+ 106us) server_negotiation.cc:407] Connection header received
0807 04:27:09.727962 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.727968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.728073 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.728190 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:09.728703 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.729588 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.730883 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.731265 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.731356 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.741309 (+ 9953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:09.741320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:09.741334 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:09.741943 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:09.741993 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:09.742014 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:09.742168 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.742327 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.742450 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 04:27:09.743549 20662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:09.782250 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.781785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53374)
0807 04:27:09.781999 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.782096 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 04:27:09.783239 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.782890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53376)
0807 04:27:09.783040 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.783122 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 04:27:09.834255 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20533
I20260807 04:27:09.844074 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.843447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53382)
0807 04:27:09.843751 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.843839 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260807 04:27:09.845113 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.844761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53386)
0807 04:27:09.844936 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.845007 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 04:27:09.854199 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20670
I20260807 04:27:09.860746 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.854923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.1.244.195:40587)
0807 04:27:09.855098 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:09.855102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:09.855188 (+ 86us) server_negotiation.cc:407] Connection header received
0807 04:27:09.855330 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:09.855333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:09.855433 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:09.855517 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:09.855959 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.856625 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.858102 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:09.858486 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:09.858580 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:09.859531 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:09.859540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:09.859555 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:09.860183 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:09.860234 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:09.860253 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:09.860336 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:09.860457 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 04:27:09.860533 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:27:09.861397 20799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:09.925920 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.925420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53398)
0807 04:27:09.925621 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.925746 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 04:27:09.927034 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:09.926623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:36705 (local address 127.0.0.1:53414)
0807 04:27:09.926824 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:09.926926 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 04:27:09.949033 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20670
I20260807 04:27:09.967578 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20298
I20260807 04:27:09.968657 20363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:10.077492 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20298
2026-08-07T04:27:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2860 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:10 dist-test-slave-zkpd krb5kdc[20861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:10 dist-test-slave-zkpd krb5kdc[20861](info): set up 2 sockets
Aug 07 04:27:10 dist-test-slave-zkpd krb5kdc[20861](info): commencing operation
krb5kdc: starting...
W20260807 04:27:11.231179 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:11 dist-test-slave-zkpd krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:11.511237 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40027
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:11.843111 20874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:11.843364 20874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:11.843449 20874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:11.850458 20874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:11.850553 20874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:11.850598 20874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:11.850656 20874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:11.850699 20874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:11.860097 20874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:11.861198 20874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:11.862386 20874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:11.873561 20879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:11.873593 20880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:11.873698 20882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:11.874789 20874 server_base.cc:1034] running on GCE node
I20260807 04:27:11.875489 20874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:11.876816 20874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:11.878057 20874 hybrid_clock.cc:648] HybridClock initialized: now 1786076831878018 us; error 45 us; skew 500 ppm
I20260807 04:27:11.878391 20874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:11 dist-test-slave-zkpd krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076831, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:11.881414 20874 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:11.882855 20874 webserver.cc:460] Webserver started at http://127.1.244.254:39265/ using document root <none> and password file <none>
I20260807 04:27:11.883505 20874 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:11.883651 20874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:11.884024 20874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:11.886859 20874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "8cb42b4949894ee79d3faed50135e09c"
format_stamp: "Formatted at 2026-08-07 04:27:11 on dist-test-slave-zkpd"
server_key: "f3bb65ae930bb58f7d7b9458e643513f"
server_key_iv: "92b5a3b9aa31bb8d888bd4c806546781"
server_key_version: "encryptionkey@0"
I20260807 04:27:11.887596 20874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "8cb42b4949894ee79d3faed50135e09c"
format_stamp: "Formatted at 2026-08-07 04:27:11 on dist-test-slave-zkpd"
server_key: "f3bb65ae930bb58f7d7b9458e643513f"
server_key_iv: "92b5a3b9aa31bb8d888bd4c806546781"
server_key_version: "encryptionkey@0"
I20260807 04:27:11.892513 20874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 04:27:11.895784 20889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:11.897050 20874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:27:11.897225 20874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "8cb42b4949894ee79d3faed50135e09c"
format_stamp: "Formatted at 2026-08-07 04:27:11 on dist-test-slave-zkpd"
server_key: "f3bb65ae930bb58f7d7b9458e643513f"
server_key_iv: "92b5a3b9aa31bb8d888bd4c806546781"
server_key_version: "encryptionkey@0"
I20260807 04:27:11.897418 20874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:11.914796 20874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:11.918453 20874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:11.918730 20874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:11.936944 20874 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40027
I20260807 04:27:11.936964 20941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40027 every 8 connection(s)
I20260807 04:27:11.938251 20874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:11.942287 20943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:11.946851 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20874
I20260807 04:27:11.947053 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:11.947495 2003 external_mini_cluster.cc:1428] Setting key d9914f84b9219fa55751be72cc697b15
I20260807 04:27:11.952972 20943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c: Bootstrap starting.
I20260807 04:27:11.956604 20943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:11.957873 20943 log.cc:824] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:11.960919 20943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c: No bootstrap required, opened a new log
Aug 07 04:27:11 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:11.971282 20943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb42b4949894ee79d3faed50135e09c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40027 } }
I20260807 04:27:11.971724 20943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:11.971844 20943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb42b4949894ee79d3faed50135e09c, State: Initialized, Role: FOLLOWER
I20260807 04:27:11.972533 20943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb42b4949894ee79d3faed50135e09c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40027 } }
I20260807 04:27:11.972744 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:11.949540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.1.244.254:40027)
0807 04:27:11.950420 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:11.950431 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:11.950473 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:27:11.952257 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:11.952278 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:11.952635 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:11.952934 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:11.954884 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:11.956231 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:11.958150 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:11.959193 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:11.959399 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:11.964021 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:11.964032 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:11.964050 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:11.964110 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:11.968183 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:11.968879 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:11.968887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:11.968896 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:11.969118 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:11.969471 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:11.969476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:11.969480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:11.970188 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:11.970498 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:11.970871 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 04:27:11.971307 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":415,"threads_started":1}
I20260807 04:27:11.972836 20943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:11.972971 20943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:11.973155 20943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:11.976006 20943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb42b4949894ee79d3faed50135e09c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40027 } }
I20260807 04:27:11.976435 20943 leader_election.cc:302] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cb42b4949894ee79d3faed50135e09c; no voters:
I20260807 04:27:11.976955 20943 leader_election.cc:288] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:11.977103 20948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:11.977821 20948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [term 1 LEADER]: Becoming Leader. State: Replica: 8cb42b4949894ee79d3faed50135e09c, State: Running, Role: LEADER
I20260807 04:27:11.978443 20948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb42b4949894ee79d3faed50135e09c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40027 } }
I20260807 04:27:11.979010 20943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:11.981302 20950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cb42b4949894ee79d3faed50135e09c. Latest consensus state: current_term: 1 leader_uuid: "8cb42b4949894ee79d3faed50135e09c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb42b4949894ee79d3faed50135e09c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40027 } } }
I20260807 04:27:11.981746 20950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:11.983283 20955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:11.983059 20949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cb42b4949894ee79d3faed50135e09c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb42b4949894ee79d3faed50135e09c" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40027 } } }
I20260807 04:27:11.983398 20949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:11.988559 20955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:11.997747 20955 catalog_manager.cc:1347] Generated new cluster ID: 45221aec6ea6461e9a1204d4bcdb0084
I20260807 04:27:11.997860 20955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:12.010874 20955 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:12.012144 20955 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:12.018880 20955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c: Generated new TSK 0
I20260807 04:27:12.019752 20955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:27:12.096956 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40027
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:12.439226 20971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:12.439534 20971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:12.439692 20971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:12.446892 20971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:12.447046 20971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:12.447229 20971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:27:12.456784 20971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:12.457980 20971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:12.459347 20971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:12.473214 20976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:12.473122 20977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:12.473178 20979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:12.473897 20971 server_base.cc:1034] running on GCE node
I20260807 04:27:12.474588 20971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:12.475504 20971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:12.476729 20971 hybrid_clock.cc:648] HybridClock initialized: now 1786076832476692 us; error 45 us; skew 500 ppm
I20260807 04:27:12.477044 20971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:12 dist-test-slave-zkpd krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076832, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:12.480309 20971 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:27:12.481727 20971 webserver.cc:460] Webserver started at http://127.1.244.193:39921/ using document root <none> and password file <none>
I20260807 04:27:12.482378 20971 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:12.482547 20971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:12.482899 20971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:12.486038 20971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "5845af77b74b49388b1ccbfad11ebd69"
format_stamp: "Formatted at 2026-08-07 04:27:12 on dist-test-slave-zkpd"
server_key: "4cc158c29ddcb375667e698b3b864b9d"
server_key_iv: "8fdfefa3cb1129cfa049642c738cd360"
server_key_version: "encryptionkey@0"
I20260807 04:27:12.486888 20971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "5845af77b74b49388b1ccbfad11ebd69"
format_stamp: "Formatted at 2026-08-07 04:27:12 on dist-test-slave-zkpd"
server_key: "4cc158c29ddcb375667e698b3b864b9d"
server_key_iv: "8fdfefa3cb1129cfa049642c738cd360"
server_key_version: "encryptionkey@0"
I20260807 04:27:12.491968 20971 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 04:27:12.495180 20986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:12.496502 20971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 04:27:12.496673 20971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "5845af77b74b49388b1ccbfad11ebd69"
format_stamp: "Formatted at 2026-08-07 04:27:12 on dist-test-slave-zkpd"
server_key: "4cc158c29ddcb375667e698b3b864b9d"
server_key_iv: "8fdfefa3cb1129cfa049642c738cd360"
server_key_version: "encryptionkey@0"
I20260807 04:27:12.496866 20971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:12.523648 20971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:12.527694 20971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:12.528005 20971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:12.528880 20971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:12.530107 20971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:12.530180 20971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:12.530263 20971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:12.530308 20971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:12.562970 20971 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:43333
I20260807 04:27:12.563019 21105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:43333 every 8 connection(s)
I20260807 04:27:12.564321 20971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:27:12.565829 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 20971
I20260807 04:27:12.566049 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:27:12.566507 2003 external_mini_cluster.cc:1428] Setting key 66eb72e8b7f6995f4c5443a111ac61b7
Aug 07 04:27:12 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076832, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:12.587692 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:12.566947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:39967 (local address 127.1.244.254:40027)
0807 04:27:12.567480 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:12.567489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:12.568525 (+ 1036us) server_negotiation.cc:407] Connection header received
0807 04:27:12.571290 (+ 2765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:12.571294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:12.571362 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:12.571449 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:12.573559 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:12.574580 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:12.577052 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:12.577679 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:12.577800 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:12.581935 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:12.581940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:12.581943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:12.581974 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:12.584725 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:12.585428 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:12.585433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:12.585436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:12.585548 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:12.586008 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:12.586013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:12.586016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:12.586258 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:12.586419 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:12.587088 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 04:27:12.587305 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":272,"threads_started":1}
I20260807 04:27:12.588630 21109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:12.567549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40027 (local address 127.1.244.193:39967)
0807 04:27:12.568362 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:12.568435 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:12.570990 (+ 2555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:12.571723 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:12.571737 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:12.572578 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:12.573366 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:12.573395 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:12.574757 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:12.574765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:12.576907 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:12.576926 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:12.577913 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:12.577923 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:12.578244 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:12.579005 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:12.579039 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:12.581717 (+ 2678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:12.584867 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:12.584875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:12.584890 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:12.585294 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:12.585657 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:12.585662 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:12.585666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:12.585875 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:12.586389 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:12.586396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:12.586719 (+ 323us) client_negotiation.cc:769] Sending connection context
0807 04:27:12.586975 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 04:27:12.587458 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":329,"threads_started":1}
I20260807 04:27:12.590225 21106 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40027
I20260807 04:27:12.590507 21106 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:12.591207 21106 heartbeater.cc:511] Master 127.1.244.254:40027 requested a full tablet report, sending...
I20260807 04:27:12.593240 20906 ts_manager.cc:194] Registered new tserver with Master: 5845af77b74b49388b1ccbfad11ebd69 (127.1.244.193:43333)
I20260807 04:27:12.594770 20906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:39967
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:27:12.631549 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40027
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:12.960412 21112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:12.960662 21112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:12.960754 21112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:12.967691 21112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:12.967782 21112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:12.967968 21112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:27:12.977412 21112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:12.978572 21112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:12.979974 21112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:12.993369 21120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:12.993484 21118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:12.993734 21117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:12.994534 21112 server_base.cc:1034] running on GCE node
I20260807 04:27:12.995115 21112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:12.996035 21112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:12.997229 21112 hybrid_clock.cc:648] HybridClock initialized: now 1786076832997191 us; error 36 us; skew 500 ppm
I20260807 04:27:12.997588 21112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:12 dist-test-slave-zkpd krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076832, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:13.000731 21112 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:27:13.002136 21112 webserver.cc:460] Webserver started at http://127.1.244.194:39841/ using document root <none> and password file <none>
I20260807 04:27:13.002795 21112 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:13.002945 21112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:13.003252 21112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:13.006093 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "5a8e3c98929d4a92b937ca08511f0d09"
format_stamp: "Formatted at 2026-08-07 04:27:12 on dist-test-slave-zkpd"
server_key: "df2fa92a076407003425ec0ebded4b78"
server_key_iv: "d39d89fbfacf55713b029e376388bd12"
server_key_version: "encryptionkey@0"
I20260807 04:27:13.006873 21112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "5a8e3c98929d4a92b937ca08511f0d09"
format_stamp: "Formatted at 2026-08-07 04:27:12 on dist-test-slave-zkpd"
server_key: "df2fa92a076407003425ec0ebded4b78"
server_key_iv: "d39d89fbfacf55713b029e376388bd12"
server_key_version: "encryptionkey@0"
I20260807 04:27:13.011873 21112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:27:13.015071 21128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:13.016283 21112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260807 04:27:13.016456 21112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "5a8e3c98929d4a92b937ca08511f0d09"
format_stamp: "Formatted at 2026-08-07 04:27:12 on dist-test-slave-zkpd"
server_key: "df2fa92a076407003425ec0ebded4b78"
server_key_iv: "d39d89fbfacf55713b029e376388bd12"
server_key_version: "encryptionkey@0"
I20260807 04:27:13.016650 21112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:13.034040 21112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:13.037889 21112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:13.038205 21112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:13.039122 21112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:13.040472 21112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:13.040544 21112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:13.040721 21112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:13.040774 21112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:13.073819 21112 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:41169
I20260807 04:27:13.073911 21248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:41169 every 8 connection(s)
I20260807 04:27:13.075109 21112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:27:13.078961 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 21112
I20260807 04:27:13.079159 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:27:13.079630 2003 external_mini_cluster.cc:1428] Setting key f50583002d4e2d2a1e0fc62497c76152
Aug 07 04:27:13 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076832, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:13.096442 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.077718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44105 (local address 127.1.244.254:40027)
0807 04:27:13.077909 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.077914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.079070 (+ 1156us) server_negotiation.cc:407] Connection header received
0807 04:27:13.081295 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.081299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.081361 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.081487 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:13.083081 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.083776 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.085607 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.086122 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.086274 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.089728 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:13.089736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:13.089744 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:13.089788 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:13.093396 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:13.094110 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:13.094121 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:13.094125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:13.094254 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:13.094697 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:13.094700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:13.094703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:13.094975 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:13.095135 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.096023 (+ 888us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.096192 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:27:13.097872 21251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.078100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40027 (local address 127.1.244.194:44105)
0807 04:27:13.078898 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.078959 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.081009 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.081760 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.081773 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.082315 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:13.082903 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.082928 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.083958 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.083968 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.085443 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.085458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.086358 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.086365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.086642 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.087312 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:13.087339 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:13.089509 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:13.093523 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:13.093530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:13.093541 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:13.093969 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:13.094367 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:13.094373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:13.094376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:13.094568 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:13.095105 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:13.095114 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:13.095566 (+ 452us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.095867 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.096460 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":287,"threads_started":1}
I20260807 04:27:13.099328 21249 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40027
I20260807 04:27:13.099627 21249 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:13.100459 21249 heartbeater.cc:511] Master 127.1.244.254:40027 requested a full tablet report, sending...
I20260807 04:27:13.102131 20906 ts_manager.cc:194] Registered new tserver with Master: 5a8e3c98929d4a92b937ca08511f0d09 (127.1.244.194:41169)
I20260807 04:27:13.102979 20906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:44105
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:27:13.146551 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40027
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:13.484563 21254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:13.484905 21254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:13.485074 21254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:13.492880 21254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:13.493021 21254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:13.493194 21254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:27:13.503230 21254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:40027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:13.504632 21254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:13.505903 21254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:13.518620 21262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:13.518644 21260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:13.518779 21259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:13.519526 21254 server_base.cc:1034] running on GCE node
I20260807 04:27:13.520201 21254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:13.521153 21254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:13.522399 21254 hybrid_clock.cc:648] HybridClock initialized: now 1786076833522335 us; error 50 us; skew 500 ppm
I20260807 04:27:13.522713 21254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:13 dist-test-slave-zkpd krb5kdc[20861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076833, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:13.525861 21254 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:27:13.527245 21254 webserver.cc:460] Webserver started at http://127.1.244.195:35943/ using document root <none> and password file <none>
I20260807 04:27:13.527855 21254 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:13.528020 21254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:13.528285 21254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:13.530980 21254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "fc03c6860796489aa8451758874098e0"
format_stamp: "Formatted at 2026-08-07 04:27:13 on dist-test-slave-zkpd"
server_key: "31d81dac3df14320d94e0142b5172277"
server_key_iv: "3c1a11f85dd1801c4ae38978aa849a09"
server_key_version: "encryptionkey@0"
I20260807 04:27:13.531711 21254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "fc03c6860796489aa8451758874098e0"
format_stamp: "Formatted at 2026-08-07 04:27:13 on dist-test-slave-zkpd"
server_key: "31d81dac3df14320d94e0142b5172277"
server_key_iv: "3c1a11f85dd1801c4ae38978aa849a09"
server_key_version: "encryptionkey@0"
I20260807 04:27:13.536700 21254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:27:13.539856 21269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:13.541126 21254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:13.541256 21254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "fc03c6860796489aa8451758874098e0"
format_stamp: "Formatted at 2026-08-07 04:27:13 on dist-test-slave-zkpd"
server_key: "31d81dac3df14320d94e0142b5172277"
server_key_iv: "3c1a11f85dd1801c4ae38978aa849a09"
server_key_version: "encryptionkey@0"
I20260807 04:27:13.541406 21254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:13.561646 21254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:13.565869 21254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:13.566197 21254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:13.567312 21254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:13.568624 21254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:13.568727 21254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:13.568881 21254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:13.568961 21254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:13.597561 21106 heartbeater.cc:503] Master 127.1.244.254:40027 was elected leader, sending a full tablet report...
I20260807 04:27:13.604219 21254 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39979
I20260807 04:27:13.604298 21385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39979 every 8 connection(s)
I20260807 04:27:13.605470 21254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:27:13.614158 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 21254
I20260807 04:27:13.614373 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:27:13.614908 2003 external_mini_cluster.cc:1428] Setting key 1bf2378617db690af3642b689f3d085d
Aug 07 04:27:13 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076833, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:13.625365 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.607785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:34231 (local address 127.1.244.254:40027)
0807 04:27:13.608318 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.608322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.609114 (+ 792us) server_negotiation.cc:407] Connection header received
0807 04:27:13.610922 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.610928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.611016 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.611134 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:13.612643 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.613409 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.615653 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.616306 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.616422 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.620167 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:13.620171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:13.620174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:13.620206 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:13.622758 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:13.623398 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:13.623401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:13.623404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:13.623523 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:13.623956 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:13.623959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:13.623961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:13.624207 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:13.624319 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.624913 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.625088 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":314,"threads_started":1}
I20260807 04:27:13.626273 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.608285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40027 (local address 127.1.244.195:34231)
0807 04:27:13.608961 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.609004 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.610666 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.611370 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.611378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.611879 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:13.612481 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.612497 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.613549 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.613561 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.615480 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.615492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.616564 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.616573 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.616871 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.617590 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:13.617641 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:13.619971 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:13.622883 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:13.622889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:13.622899 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:13.623290 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:13.623614 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:13.623616 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:13.623619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:13.623788 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:13.624312 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:13.624318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:13.624610 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.624816 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.625191 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":262,"threads_started":1}
I20260807 04:27:13.627664 21386 heartbeater.cc:347] Connected to a master server at 127.1.244.254:40027
I20260807 04:27:13.627996 21386 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:13.628839 21386 heartbeater.cc:511] Master 127.1.244.254:40027 requested a full tablet report, sending...
I20260807 04:27:13.630440 20906 ts_manager.cc:194] Registered new tserver with Master: fc03c6860796489aa8451758874098e0 (127.1.244.195:39979)
I20260807 04:27:13.631196 20906 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:34231
I20260807 04:27:13.642303 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:27:13.658991 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.646819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.1.244.254:40027)
0807 04:27:13.647071 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.647077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.647303 (+ 226us) server_negotiation.cc:407] Connection header received
0807 04:27:13.647429 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.647432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.647491 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.647568 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:13.649046 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.649775 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.651102 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.651530 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.651624 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.654086 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:13.654090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:13.654093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:13.654122 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:13.656529 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:13.657209 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:13.657213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:13.657215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:13.657331 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:13.657701 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:13.657704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:13.657705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:13.657985 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:13.658174 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.658474 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.658607 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 04:27:13.662708 20906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:27:13.669649 20906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:27:13.702678 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.681476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41169 (local address 127.0.0.1:53328)
0807 04:27:13.682722 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.682777 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.682961 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.695717 (+ 12756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.695722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.695755 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:13.696112 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.696121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.697477 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.697489 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.699743 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.699760 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.701622 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.701630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.701836 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.701895 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.702006 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.702198 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":252,"thread_start_us":12504,"threads_started":1}
I20260807 04:27:13.703119 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.681747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:43333 (local address 127.0.0.1:38170)
0807 04:27:13.693712 (+ 11965us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.693746 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.693893 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.694643 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.694650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.694689 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:13.695109 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.695123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.697179 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.697184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.700236 (+ 3052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.700253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.702165 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.702173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.702371 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.702431 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.702536 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.702688 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11796,"mutex_wait_us":311,"thread_start_us":1025,"threads_started":1}
I20260807 04:27:13.703253 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.682893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.1.244.195:39979)
0807 04:27:13.683734 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.683747 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.694123 (+ 10376us) server_negotiation.cc:407] Connection header received
0807 04:27:13.694513 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.694523 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.694779 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.694979 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:13.695965 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.697444 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.700490 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.701752 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.701872 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.702118 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.702720 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.702892 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":440,"threads_started":1}
I20260807 04:27:13.703575 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.682644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.1.244.193:43333)
0807 04:27:13.693875 (+ 11231us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.693887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.693940 (+ 53us) server_negotiation.cc:407] Connection header received
0807 04:27:13.694077 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.694088 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.694388 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.694639 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:13.695303 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.697035 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.700442 (+ 3407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.701947 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.702080 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.702291 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.702608 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.702991 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9602,"thread_start_us":9463,"threads_started":1}
I20260807 04:27:13.704002 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.682185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39979 (local address 127.0.0.1:50278)
0807 04:27:13.694012 (+ 11827us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.694033 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.694342 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.695267 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.695272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.695296 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:13.695687 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.695700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.697553 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.697562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.700316 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.700330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.702302 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.702308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.702508 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.702542 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.703449 (+ 907us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.703602 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11702,"mutex_wait_us":392,"thread_start_us":11956,"threads_started":1}
I20260807 04:27:13.704432 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.682269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.1.244.194:41169)
0807 04:27:13.695282 (+ 13013us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.695290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.695321 (+ 31us) server_negotiation.cc:407] Connection header received
0807 04:27:13.695397 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.695402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.695540 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.695727 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:13.696265 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.697322 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.699934 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.701400 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.703029 (+ 1629us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.703401 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.703489 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.703963 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12874,"thread_start_us":12769,"threads_started":1}
I20260807 04:27:13.707234 21320 tablet_service.cc:1467] Processing CreateTablet for tablet 16b0f0f3b56b4809ab2d79b73e081e93 (DEFAULT_TABLE table=test-table [id=4bfb72a5e1c9434ea428ea5757177213]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:13.707484 21038 tablet_service.cc:1467] Processing CreateTablet for tablet 16b0f0f3b56b4809ab2d79b73e081e93 (DEFAULT_TABLE table=test-table [id=4bfb72a5e1c9434ea428ea5757177213]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:13.707603 21181 tablet_service.cc:1467] Processing CreateTablet for tablet 16b0f0f3b56b4809ab2d79b73e081e93 (DEFAULT_TABLE table=test-table [id=4bfb72a5e1c9434ea428ea5757177213]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:13.708659 21320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16b0f0f3b56b4809ab2d79b73e081e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:13.708948 21038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16b0f0f3b56b4809ab2d79b73e081e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:13.708987 21181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16b0f0f3b56b4809ab2d79b73e081e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:13.721203 21408 tablet_bootstrap.cc:492] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Bootstrap starting.
I20260807 04:27:13.724632 21410 tablet_bootstrap.cc:492] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Bootstrap starting.
I20260807 04:27:13.724941 21408 tablet_bootstrap.cc:654] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:13.726347 21408 log.cc:824] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:13.727799 21409 tablet_bootstrap.cc:492] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Bootstrap starting.
I20260807 04:27:13.728332 21410 tablet_bootstrap.cc:654] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:13.729506 21408 tablet_bootstrap.cc:492] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: No bootstrap required, opened a new log
I20260807 04:27:13.729599 21410 log.cc:824] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:13.729833 21408 ts_tablet_manager.cc:1397] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:27:13.731338 21409 tablet_bootstrap.cc:654] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:13.732448 21409 log.cc:824] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:13.732653 21410 tablet_bootstrap.cc:492] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: No bootstrap required, opened a new log
I20260807 04:27:13.732903 21410 ts_tablet_manager.cc:1397] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 04:27:13.734926 21409 tablet_bootstrap.cc:492] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: No bootstrap required, opened a new log
I20260807 04:27:13.735181 21409 ts_tablet_manager.cc:1397] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:27:13.740522 21408 raft_consensus.cc:348] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.740949 21408 raft_consensus.cc:374] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:13.741045 21408 raft_consensus.cc:729] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc03c6860796489aa8451758874098e0, State: Initialized, Role: FOLLOWER
I20260807 04:27:13.741676 21408 consensus_queue.cc:260] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.742166 21409 raft_consensus.cc:348] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.742594 21409 raft_consensus.cc:374] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:13.742698 21409 raft_consensus.cc:729] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a8e3c98929d4a92b937ca08511f0d09, State: Initialized, Role: FOLLOWER
I20260807 04:27:13.742935 21386 heartbeater.cc:503] Master 127.1.244.254:40027 was elected leader, sending a full tablet report...
I20260807 04:27:13.743484 21408 ts_tablet_manager.cc:1428] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 04:27:13.743297 21410 raft_consensus.cc:348] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.743391 21409 consensus_queue.cc:260] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.743753 21410 raft_consensus.cc:374] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:13.743854 21410 raft_consensus.cc:729] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5845af77b74b49388b1ccbfad11ebd69, State: Initialized, Role: FOLLOWER
I20260807 04:27:13.744454 21410 consensus_queue.cc:260] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.750326 21249 heartbeater.cc:503] Master 127.1.244.254:40027 was elected leader, sending a full tablet report...
I20260807 04:27:13.751047 21409 ts_tablet_manager.cc:1428] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260807 04:27:13.752632 21410 ts_tablet_manager.cc:1428] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Time spent starting tablet: real 0.020s user 0.011s sys 0.009s
W20260807 04:27:13.816833 21107 tablet.cc:2369] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:27:13.827140 21250 tablet.cc:2369] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:27:13.857110 21387 tablet.cc:2369] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:13.910341 21417 raft_consensus.cc:482] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:13.910624 21417 raft_consensus.cc:504] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.912279 21417 leader_election.cc:288] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a8e3c98929d4a92b937ca08511f0d09 (127.1.244.194:41169), fc03c6860796489aa8451758874098e0 (127.1.244.195:39979)
I20260807 04:27:13.919086 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.912755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:55847 (local address 127.1.244.194:41169)
0807 04:27:13.912962 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.912968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.913753 (+ 785us) server_negotiation.cc:407] Connection header received
0807 04:27:13.914002 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.914006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.914068 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.914181 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:13.914764 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.915489 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.917363 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.918149 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.918322 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.918414 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.918701 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.918825 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:27:13.919256 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.912569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:41169 (local address 127.1.244.193:55847)
0807 04:27:13.913589 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.913616 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.913806 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.914198 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.914202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.914227 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:13.914613 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.914622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.915618 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.915625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.917202 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.917218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.918388 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.918394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.918544 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.918572 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.918676 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.918804 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":52,"thread_start_us":918,"threads_started":1}
I20260807 04:27:13.919283 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.912713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39979 (local address 127.1.244.193:51699)
0807 04:27:13.913805 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:13.913821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:13.913942 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:13.914299 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:13.914303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:13.914343 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:13.914754 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.914763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.915772 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.915778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.917256 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:13.917268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.918443 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.918450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:13.918594 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.918617 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:27:13.918717 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 04:27:13.918839 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":19,"thread_start_us":485,"threads_started":1}
I20260807 04:27:13.919126 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.912752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51699 (local address 127.1.244.195:39979)
0807 04:27:13.912950 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.912958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.913905 (+ 947us) server_negotiation.cc:407] Connection header received
0807 04:27:13.914112 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.914116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.914177 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.914276 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:13.914911 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.915627 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.917427 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.918156 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.918352 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.918446 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.918752 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.918873 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 04:27:13.920334 21202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16b0f0f3b56b4809ab2d79b73e081e93" candidate_uuid: "5845af77b74b49388b1ccbfad11ebd69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a8e3c98929d4a92b937ca08511f0d09" is_pre_election: true
I20260807 04:27:13.920544 21341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16b0f0f3b56b4809ab2d79b73e081e93" candidate_uuid: "5845af77b74b49388b1ccbfad11ebd69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc03c6860796489aa8451758874098e0" is_pre_election: true
I20260807 04:27:13.920956 21202 raft_consensus.cc:2454] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5845af77b74b49388b1ccbfad11ebd69 in term 0.
I20260807 04:27:13.921020 21341 raft_consensus.cc:2454] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5845af77b74b49388b1ccbfad11ebd69 in term 0.
I20260807 04:27:13.921707 20988 leader_election.cc:302] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845af77b74b49388b1ccbfad11ebd69, fc03c6860796489aa8451758874098e0; no voters:
I20260807 04:27:13.922114 21417 raft_consensus.cc:2781] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:27:13.922258 21417 raft_consensus.cc:482] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:27:13.922374 21417 raft_consensus.cc:3037] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:13.924611 21417 raft_consensus.cc:504] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.925225 21417 leader_election.cc:288] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [CANDIDATE]: Term 1 election: Requested vote from peers 5a8e3c98929d4a92b937ca08511f0d09 (127.1.244.194:41169), fc03c6860796489aa8451758874098e0 (127.1.244.195:39979)
I20260807 04:27:13.925639 21202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16b0f0f3b56b4809ab2d79b73e081e93" candidate_uuid: "5845af77b74b49388b1ccbfad11ebd69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a8e3c98929d4a92b937ca08511f0d09"
I20260807 04:27:13.925719 21341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16b0f0f3b56b4809ab2d79b73e081e93" candidate_uuid: "5845af77b74b49388b1ccbfad11ebd69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc03c6860796489aa8451758874098e0"
I20260807 04:27:13.925841 21202 raft_consensus.cc:3037] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:13.925959 21341 raft_consensus.cc:3037] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:13.928121 21202 raft_consensus.cc:2454] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5845af77b74b49388b1ccbfad11ebd69 in term 1.
I20260807 04:27:13.928267 21341 raft_consensus.cc:2454] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5845af77b74b49388b1ccbfad11ebd69 in term 1.
I20260807 04:27:13.928576 20989 leader_election.cc:302] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5845af77b74b49388b1ccbfad11ebd69, 5a8e3c98929d4a92b937ca08511f0d09; no voters:
I20260807 04:27:13.928877 21417 raft_consensus.cc:2781] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:13.929342 21417 raft_consensus.cc:686] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 1 LEADER]: Becoming Leader. State: Replica: 5845af77b74b49388b1ccbfad11ebd69, State: Running, Role: LEADER
I20260807 04:27:13.929787 21417 consensus_queue.cc:237] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } }
I20260807 04:27:13.934028 20906 catalog_manager.cc:5495] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5845af77b74b49388b1ccbfad11ebd69 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "5845af77b74b49388b1ccbfad11ebd69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5845af77b74b49388b1ccbfad11ebd69" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 43333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 } health_report { overall_health: UNKNOWN } } }
I20260807 04:27:13.992084 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:13.986650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.1.244.193:43333)
0807 04:27:13.986920 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:13.986928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:13.986951 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:27:13.987058 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:13.987061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:13.987136 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:13.987293 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:27:13.987703 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.988465 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.989802 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:13.990204 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:13.990350 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:13.990923 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:27:13.991416 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:27:13.991599 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:13.991758 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 04:27:13.991805 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 04:27:14.000229 21202 raft_consensus.cc:1261] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 1 FOLLOWER]: Refusing update from remote peer 5845af77b74b49388b1ccbfad11ebd69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:14.000276 21341 raft_consensus.cc:1261] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 1 FOLLOWER]: Refusing update from remote peer 5845af77b74b49388b1ccbfad11ebd69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:14.001237 21421 consensus_queue.cc:1035] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a8e3c98929d4a92b937ca08511f0d09" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 41169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:14.001708 21417 consensus_queue.cc:1035] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc03c6860796489aa8451758874098e0" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:14.026685 20906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:14.026914 20906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:14.030565 20906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cb42b4949894ee79d3faed50135e09c: Sending DeleteTablet for 3 replicas of tablet 16b0f0f3b56b4809ab2d79b73e081e93
I20260807 04:27:14.031644 21181 tablet_service.cc:1514] Processing DeleteTablet for tablet 16b0f0f3b56b4809ab2d79b73e081e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:53328
I20260807 04:27:14.031762 21038 tablet_service.cc:1514] Processing DeleteTablet for tablet 16b0f0f3b56b4809ab2d79b73e081e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:38170
I20260807 04:27:14.032178 21320 tablet_service.cc:1514] Processing DeleteTablet for tablet 16b0f0f3b56b4809ab2d79b73e081e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:14 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:50278
I20260807 04:27:14.037098 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20971
I20260807 04:27:14.043474 21435 tablet_replica.cc:330] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: stopping tablet replica
I20260807 04:27:14.044054 21435 raft_consensus.cc:2229] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:27:14.046569 21435 raft_consensus.cc:2258] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:14.047372 21436 tablet_replica.cc:330] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: stopping tablet replica
I20260807 04:27:14.047792 21436 raft_consensus.cc:2229] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:14.048370 21436 raft_consensus.cc:2258] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:14.048669 21434 tablet_replica.cc:330] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: stopping tablet replica
I20260807 04:27:14.049047 21434 raft_consensus.cc:2229] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:14.049504 21434 raft_consensus.cc:2258] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:14.050820 21435 ts_tablet_manager.cc:1905] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:14.051538 21434 ts_tablet_manager.cc:1905] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:14.058524 21435 ts_tablet_manager.cc:1918] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:14.058518 21434 ts_tablet_manager.cc:1918] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:14.058691 21435 log.cc:1196] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/16b0f0f3b56b4809ab2d79b73e081e93
Aug 07 04:27:14 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:27:14.058715 21434 log.cc:1196] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/16b0f0f3b56b4809ab2d79b73e081e93
I20260807 04:27:14.059293 21434 ts_tablet_manager.cc:1939] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5a8e3c98929d4a92b937ca08511f0d09: Deleting consensus metadata
I20260807 04:27:14.059309 21435 ts_tablet_manager.cc:1939] T 16b0f0f3b56b4809ab2d79b73e081e93 P 5845af77b74b49388b1ccbfad11ebd69: Deleting consensus metadata
I20260807 04:27:14.061183 20892 catalog_manager.cc:4841] TS 5a8e3c98929d4a92b937ca08511f0d09 (127.1.244.194:41169): tablet 16b0f0f3b56b4809ab2d79b73e081e93 (table test-table [id=4bfb72a5e1c9434ea428ea5757177213]) successfully deleted
I20260807 04:27:14.061971 21436 ts_tablet_manager.cc:1905] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:14.062741 20893 catalog_manager.cc:4841] TS 5845af77b74b49388b1ccbfad11ebd69 (127.1.244.193:43333): tablet 16b0f0f3b56b4809ab2d79b73e081e93 (table test-table [id=4bfb72a5e1c9434ea428ea5757177213]) successfully deleted
I20260807 04:27:14.066833 21436 ts_tablet_manager.cc:1918] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:14.067006 21436 log.cc:1196] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/16b0f0f3b56b4809ab2d79b73e081e93
I20260807 04:27:14.067426 21436 ts_tablet_manager.cc:1939] T 16b0f0f3b56b4809ab2d79b73e081e93 P fc03c6860796489aa8451758874098e0: Deleting consensus metadata
I20260807 04:27:14.068293 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:14.038429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.1.244.193:43333)
0807 04:27:14.038616 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:14.038622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:14.047700 (+ 9078us) server_negotiation.cc:407] Connection header received
0807 04:27:14.047933 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:14.047939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:14.048021 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:14.048127 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:14.050285 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:14.051479 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:14.053525 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:14.054258 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:14.054355 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:14.060216 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:14.060231 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:14.060251 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:14.060291 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:14.064548 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:14.065476 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:14.065485 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:14.065489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:14.065670 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:14.066164 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:14.066169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:14.066172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:14.066611 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:14.066912 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:14.067533 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 04:27:14.067869 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:14.069182 20891 catalog_manager.cc:4841] TS fc03c6860796489aa8451758874098e0 (127.1.244.195:39979): tablet 16b0f0f3b56b4809ab2d79b73e081e93 (table test-table [id=4bfb72a5e1c9434ea428ea5757177213]) successfully deleted
I20260807 04:27:14.069733 21101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:14.168534 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20971
I20260807 04:27:14.190945 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 21112
Aug 07 04:27:14 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:27:14.206092 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:14.191795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.1.244.194:41169)
0807 04:27:14.192015 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:14.192022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:14.192041 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:27:14.192211 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:14.192221 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:14.192284 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:14.192391 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:14.193750 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:14.194533 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:14.195802 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:14.196254 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:14.196388 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:14.200496 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:14.200511 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:14.200550 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:14.200604 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:14.203564 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:14.204266 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:14.204275 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:14.204277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:14.204436 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:14.204812 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:14.204815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:14.204817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:14.205192 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:14.205421 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:14.205667 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 04:27:14.205826 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:27:14.206760 21244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:14.296927 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 21112
I20260807 04:27:14.316205 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 21254
Aug 07 04:27:14 dist-test-slave-zkpd krb5kdc[20861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:27:14.331240 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:14.317048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50284 (local address 127.1.244.195:39979)
0807 04:27:14.317253 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:14.317259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:14.317279 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:27:14.317453 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:14.317464 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:14.317531 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:14.317655 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:14.318984 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:14.319706 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:14.321104 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:14.321518 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:14.321660 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:14.326239 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:14.326255 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:14.326271 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:14.326307 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:14.328935 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:14.329542 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:14.329550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:14.329552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:14.329684 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:14.330002 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:14.330005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:14.330007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:14.330341 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:14.330560 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:14.330811 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 04:27:14.330978 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 04:27:14.332069 21381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:14.423102 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 21254
I20260807 04:27:14.442005 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 20874
I20260807 04:27:14.443221 20937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:14.553640 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 20874
2026-08-07T04:27:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4476 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T04:27:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:14Z Disabled control of system clock
I20260807 04:27:14.641204 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42285
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:27:14.974824 21445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:14.975174 21445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:14.975320 21445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:14.982357 21445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:14.982450 21445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:14.982546 21445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:14.982658 21445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:14.982720 21445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:14.991765 21445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:42285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:42285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:14.993014 21445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:14.994784 21445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:15.006728 21454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:15.006778 21451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:15.006734 21450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:15.007335 21445 server_base.cc:1034] running on GCE node
I20260807 04:27:15.008239 21445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:15.009580 21445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:15.010783 21445 hybrid_clock.cc:648] HybridClock initialized: now 1786076835010736 us; error 45 us; skew 500 ppm
I20260807 04:27:15.011139 21445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:15.017845 21445 webserver.cc:460] Webserver started at http://127.1.244.254:36393/ using document root <none> and password file <none>
I20260807 04:27:15.018537 21445 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:15.018658 21445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:15.018999 21445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:15.021879 21445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "319bb6605753467291f4a398dbaa678d"
format_stamp: "Formatted at 2026-08-07 04:27:15 on dist-test-slave-zkpd"
server_key: "98f9981a15dacb03d0895f6d15ddd8b8"
server_key_iv: "ef41b4e032b8144a9e57160ba502608a"
server_key_version: "encryptionkey@0"
I20260807 04:27:15.022691 21445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "319bb6605753467291f4a398dbaa678d"
format_stamp: "Formatted at 2026-08-07 04:27:15 on dist-test-slave-zkpd"
server_key: "98f9981a15dacb03d0895f6d15ddd8b8"
server_key_iv: "ef41b4e032b8144a9e57160ba502608a"
server_key_version: "encryptionkey@0"
I20260807 04:27:15.027721 21445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:27:15.031013 21460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:15.032371 21445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:15.032560 21445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "319bb6605753467291f4a398dbaa678d"
format_stamp: "Formatted at 2026-08-07 04:27:15 on dist-test-slave-zkpd"
server_key: "98f9981a15dacb03d0895f6d15ddd8b8"
server_key_iv: "ef41b4e032b8144a9e57160ba502608a"
server_key_version: "encryptionkey@0"
I20260807 04:27:15.032774 21445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:15.044584 21445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:15.045342 21445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:15.045621 21445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:15.064683 21445 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:42285
I20260807 04:27:15.064702 21512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:42285 every 8 connection(s)
I20260807 04:27:15.066009 21445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:15.066721 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 21445
I20260807 04:27:15.066936 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:15.067535 2003 external_mini_cluster.cc:1428] Setting key b2d3b2303ff0e129faa375473ff7f292
I20260807 04:27:15.070528 21513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:15.080735 21516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:15.069483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.1.244.254:42285)
0807 04:27:15.070495 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:15.070512 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:15.070556 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:27:15.072293 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:15.072314 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:15.072739 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:15.073154 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:15.073718 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:15.075051 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:15.076439 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:15.077091 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:15.077283 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:15.078520 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:15.078527 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:15.078539 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:15.079093 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:15.079167 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:15.079187 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:15.079320 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:15.079547 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 04:27:15.079835 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":363,"threads_started":1}
I20260807 04:27:15.081223 21513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d: Bootstrap starting.
I20260807 04:27:15.084743 21513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:15.085697 21513 log.cc:824] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:15.088384 21513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d: No bootstrap required, opened a new log
I20260807 04:27:15.095410 21513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319bb6605753467291f4a398dbaa678d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42285 } }
I20260807 04:27:15.095746 21513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:15.095867 21513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319bb6605753467291f4a398dbaa678d, State: Initialized, Role: FOLLOWER
I20260807 04:27:15.096422 21513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319bb6605753467291f4a398dbaa678d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42285 } }
I20260807 04:27:15.096644 21513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:15.096798 21513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:15.096966 21513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:15.098990 21513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319bb6605753467291f4a398dbaa678d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42285 } }
I20260807 04:27:15.099495 21513 leader_election.cc:302] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 319bb6605753467291f4a398dbaa678d; no voters:
I20260807 04:27:15.100129 21513 leader_election.cc:288] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:15.100226 21518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:15.100989 21518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [term 1 LEADER]: Becoming Leader. State: Replica: 319bb6605753467291f4a398dbaa678d, State: Running, Role: LEADER
I20260807 04:27:15.101511 21518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319bb6605753467291f4a398dbaa678d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42285 } }
I20260807 04:27:15.102193 21513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:15.104295 21519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "319bb6605753467291f4a398dbaa678d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319bb6605753467291f4a398dbaa678d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42285 } } }
I20260807 04:27:15.104799 21519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:15.105588 21520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 319bb6605753467291f4a398dbaa678d. Latest consensus state: current_term: 1 leader_uuid: "319bb6605753467291f4a398dbaa678d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319bb6605753467291f4a398dbaa678d" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 42285 } } }
I20260807 04:27:15.105942 21520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:15.108174 21527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:15.112340 21527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:15.120647 21527 catalog_manager.cc:1347] Generated new cluster ID: 05160bdab61045f591313351c56558b8
I20260807 04:27:15.120764 21527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:15.152477 21527 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:15.154306 21527 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:15.162714 21527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d: Generated new TSK 0
I20260807 04:27:15.163699 21527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:15.178249 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42285
--builtin_ntp_servers=127.1.244.212:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:27:15.523119 21537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:15.523370 21537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:15.523528 21537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:15.530649 21537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:15.530771 21537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:15.530946 21537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:27:15.539969 21537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:15.541172 21537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:15.542896 21537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:15.556229 21543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:15.556239 21544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:15.556336 21546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:15.556967 21537 server_base.cc:1034] running on GCE node
I20260807 04:27:15.557545 21537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:15.558413 21537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:15.559643 21537 hybrid_clock.cc:648] HybridClock initialized: now 1786076835559597 us; error 37 us; skew 500 ppm
I20260807 04:27:15.559995 21537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:15.562204 21537 webserver.cc:460] Webserver started at http://127.1.244.193:37193/ using document root <none> and password file <none>
I20260807 04:27:15.562863 21537 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:15.562989 21537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:15.563308 21537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:15.566092 21537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "049b8ee501584aea85caaf4f75ea94f0"
format_stamp: "Formatted at 2026-08-07 04:27:15 on dist-test-slave-zkpd"
server_key: "699d3adb61e99e2ef89b482324bce860"
server_key_iv: "11bd658b91867b9734a499e7af452b7c"
server_key_version: "encryptionkey@0"
I20260807 04:27:15.566871 21537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "049b8ee501584aea85caaf4f75ea94f0"
format_stamp: "Formatted at 2026-08-07 04:27:15 on dist-test-slave-zkpd"
server_key: "699d3adb61e99e2ef89b482324bce860"
server_key_iv: "11bd658b91867b9734a499e7af452b7c"
server_key_version: "encryptionkey@0"
I20260807 04:27:15.571714 21537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 04:27:15.575143 21552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:15.576373 21537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:15.576542 21537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "049b8ee501584aea85caaf4f75ea94f0"
format_stamp: "Formatted at 2026-08-07 04:27:15 on dist-test-slave-zkpd"
server_key: "699d3adb61e99e2ef89b482324bce860"
server_key_iv: "11bd658b91867b9734a499e7af452b7c"
server_key_version: "encryptionkey@0"
I20260807 04:27:15.576756 21537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:15.590320 21537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:15.591050 21537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:15.591321 21537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:15.592298 21537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:15.593637 21537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:15.593763 21537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:15.593883 21537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:15.593967 21537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:15.626943 21537 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:42231
I20260807 04:27:15.627000 21669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:42231 every 8 connection(s)
I20260807 04:27:15.628314 21537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:27:15.637742 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 21537
I20260807 04:27:15.637959 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:27:15.638532 2003 external_mini_cluster.cc:1428] Setting key 43b710f14bc3b404d2b162090e96c24a
I20260807 04:27:15.641191 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:15.630874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:53573 (local address 127.1.244.254:42285)
0807 04:27:15.631333 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:15.631338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:15.632514 (+ 1176us) server_negotiation.cc:407] Connection header received
0807 04:27:15.634336 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:15.634341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:15.634428 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:15.634547 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:15.635572 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:15.636398 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:15.638001 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:15.638665 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:15.638804 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:15.640251 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:15.640256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:15.640259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:15.640275 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:15.640312 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:15.640324 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:15.640466 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:15.640763 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 04:27:15.640870 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":238,"threads_started":1}
I20260807 04:27:15.641765 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42285
--builtin_ntp_servers=127.1.244.212:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:27:15.642129 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:15.631473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42285 (local address 127.1.244.193:53573)
0807 04:27:15.632368 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:15.632404 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:15.634064 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:15.634779 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:15.634787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:15.634874 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:15.635407 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:15.635423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:15.636559 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:15.636564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:15.637795 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:15.637807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:15.638871 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:15.638877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:15.639072 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:15.639936 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:15.639940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:15.639989 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:15.640074 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:15.640467 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:15.640508 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:15.640515 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:27:15.640724 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 04:27:15.641030 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":267,"threads_started":1}
I20260807 04:27:15.643636 21670 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42285
I20260807 04:27:15.643973 21670 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:15.644874 21670 heartbeater.cc:511] Master 127.1.244.254:42285 requested a full tablet report, sending...
I20260807 04:27:15.646898 21477 ts_manager.cc:194] Registered new tserver with Master: 049b8ee501584aea85caaf4f75ea94f0 (127.1.244.193:42231)
I20260807 04:27:15.648416 21477 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:53573
W20260807 04:27:15.970464 21674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:15.970724 21674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:15.970816 21674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:15.977989 21674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:15.978077 21674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:15.978237 21674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:27:15.987087 21674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:15.988305 21674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:15.990007 21674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:16.003087 21679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:16.003129 21682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:16.003181 21680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:16.004460 21674 server_base.cc:1034] running on GCE node
I20260807 04:27:16.005113 21674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:16.006040 21674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:16.007237 21674 hybrid_clock.cc:648] HybridClock initialized: now 1786076836007201 us; error 45 us; skew 500 ppm
I20260807 04:27:16.007555 21674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:16.009723 21674 webserver.cc:460] Webserver started at http://127.1.244.194:44603/ using document root <none> and password file <none>
I20260807 04:27:16.010381 21674 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:16.010496 21674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:16.010825 21674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:16.013669 21674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "66287ddd73b041f9aa02cccc1048aaca"
format_stamp: "Formatted at 2026-08-07 04:27:16 on dist-test-slave-zkpd"
server_key: "ecdf5a1b25ed28a07065fcabb2e5aa52"
server_key_iv: "13496caddfb575ba355ca8ce3b2b0075"
server_key_version: "encryptionkey@0"
I20260807 04:27:16.014478 21674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "66287ddd73b041f9aa02cccc1048aaca"
format_stamp: "Formatted at 2026-08-07 04:27:16 on dist-test-slave-zkpd"
server_key: "ecdf5a1b25ed28a07065fcabb2e5aa52"
server_key_iv: "13496caddfb575ba355ca8ce3b2b0075"
server_key_version: "encryptionkey@0"
I20260807 04:27:16.019294 21674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:27:16.022667 21688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:16.023957 21674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:27:16.024127 21674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "66287ddd73b041f9aa02cccc1048aaca"
format_stamp: "Formatted at 2026-08-07 04:27:16 on dist-test-slave-zkpd"
server_key: "ecdf5a1b25ed28a07065fcabb2e5aa52"
server_key_iv: "13496caddfb575ba355ca8ce3b2b0075"
server_key_version: "encryptionkey@0"
I20260807 04:27:16.024339 21674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:16.039507 21674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:16.040262 21674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:16.040581 21674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:16.041502 21674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:16.042966 21674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:16.043080 21674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:16.043210 21674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:16.043303 21674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:16.077124 21674 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:44975
I20260807 04:27:16.077172 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:44975 every 8 connection(s)
I20260807 04:27:16.078413 21674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:27:16.082942 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 21674
I20260807 04:27:16.083115 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:27:16.083578 2003 external_mini_cluster.cc:1428] Setting key c6f570310fc7028a5a4fd68198cf8078
I20260807 04:27:16.086305 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42285
--builtin_ntp_servers=127.1.244.212:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:27:16.091351 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.081157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:51987 (local address 127.1.244.254:42285)
0807 04:27:16.081402 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.081409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.082952 (+ 1543us) server_negotiation.cc:407] Connection header received
0807 04:27:16.084956 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.084962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.085055 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.085166 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:16.086214 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.087021 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.088641 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.089037 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.089175 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.090534 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:16.090539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:16.090543 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:16.090560 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:16.090592 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:16.090605 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:16.090716 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.090940 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.091036 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 04:27:16.092396 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.081743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42285 (local address 127.1.244.194:51987)
0807 04:27:16.082772 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.082820 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.084693 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.085408 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.085416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.085496 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:16.086044 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.086060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.087190 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.087194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.088483 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.088496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.089270 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.089278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.089501 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.090212 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:16.090214 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:16.090253 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:16.090330 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:16.090705 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:16.090717 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:16.090726 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.090911 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.091307 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":292,"threads_started":1}
I20260807 04:27:16.093776 21808 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42285
I20260807 04:27:16.094074 21808 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:16.094789 21808 heartbeater.cc:511] Master 127.1.244.254:42285 requested a full tablet report, sending...
I20260807 04:27:16.096307 21477 ts_manager.cc:194] Registered new tserver with Master: 66287ddd73b041f9aa02cccc1048aaca (127.1.244.194:44975)
I20260807 04:27:16.097515 21477 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:51987
W20260807 04:27:16.423118 21811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:16.423367 21811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:16.423506 21811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:16.430994 21811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:16.431092 21811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:16.431223 21811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:27:16.440279 21811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:42285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:16.441561 21811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:16.443356 21811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:16.456538 21816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:16.456570 21817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:16.456642 21819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:16.457631 21811 server_base.cc:1034] running on GCE node
I20260807 04:27:16.458282 21811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:16.459224 21811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:16.460453 21811 hybrid_clock.cc:648] HybridClock initialized: now 1786076836460403 us; error 57 us; skew 500 ppm
I20260807 04:27:16.460793 21811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:16.462960 21811 webserver.cc:460] Webserver started at http://127.1.244.195:42129/ using document root <none> and password file <none>
I20260807 04:27:16.463629 21811 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:16.463775 21811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:16.464166 21811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:16.467106 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "946a0478b4ad4e5591af1b1b48622e86"
format_stamp: "Formatted at 2026-08-07 04:27:16 on dist-test-slave-zkpd"
server_key: "2b3269b4a1d87856c90060206a42f8fa"
server_key_iv: "ecd216a001b76fbc22a369c5a881837b"
server_key_version: "encryptionkey@0"
I20260807 04:27:16.468050 21811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "946a0478b4ad4e5591af1b1b48622e86"
format_stamp: "Formatted at 2026-08-07 04:27:16 on dist-test-slave-zkpd"
server_key: "2b3269b4a1d87856c90060206a42f8fa"
server_key_iv: "ecd216a001b76fbc22a369c5a881837b"
server_key_version: "encryptionkey@0"
I20260807 04:27:16.473024 21811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:27:16.476517 21825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:16.477671 21811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:27:16.477880 21811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "946a0478b4ad4e5591af1b1b48622e86"
format_stamp: "Formatted at 2026-08-07 04:27:16 on dist-test-slave-zkpd"
server_key: "2b3269b4a1d87856c90060206a42f8fa"
server_key_iv: "ecd216a001b76fbc22a369c5a881837b"
server_key_version: "encryptionkey@0"
I20260807 04:27:16.478091 21811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:16.492556 21811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:16.493290 21811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:16.493594 21811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:16.494513 21811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:16.495903 21811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:16.496088 21811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:16.496223 21811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:16.496316 21811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:16.529538 21811 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:39079
I20260807 04:27:16.529593 21942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:39079 every 8 connection(s)
I20260807 04:27:16.530908 21811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:27:16.537084 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 21811
I20260807 04:27:16.537285 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:27:16.537801 2003 external_mini_cluster.cc:1428] Setting key 0118439e8bf2527ce32a4a0a4068d2d0
I20260807 04:27:16.543411 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.533492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:34961 (local address 127.1.244.254:42285)
0807 04:27:16.533696 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.533701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.535036 (+ 1335us) server_negotiation.cc:407] Connection header received
0807 04:27:16.536886 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.536894 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.537007 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.537119 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:16.538361 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.539295 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.541115 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.541528 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.541623 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.542787 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:16.542791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:16.542794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:16.542810 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:16.542841 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:16.542853 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:16.542929 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.543116 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.543185 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:27:16.544417 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.534024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:42285 (local address 127.1.244.195:34961)
0807 04:27:16.534885 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.534928 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.536575 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.537437 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.537454 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.537538 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:16.538163 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.538182 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.539476 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.539481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.540990 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.541003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.541706 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.541711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.541917 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.542514 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:16.542517 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:16.542553 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:16.542621 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:16.542934 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:16.542942 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:16.542947 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.543089 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.543355 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":254,"threads_started":1}
I20260807 04:27:16.545676 21943 heartbeater.cc:347] Connected to a master server at 127.1.244.254:42285
I20260807 04:27:16.545974 21943 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:16.546706 21943 heartbeater.cc:511] Master 127.1.244.254:42285 requested a full tablet report, sending...
I20260807 04:27:16.548133 21477 ts_manager.cc:194] Registered new tserver with Master: 946a0478b4ad4e5591af1b1b48622e86 (127.1.244.195:39079)
I20260807 04:27:16.548859 21477 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:34961
I20260807 04:27:16.552935 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:27:16.562834 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.557162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46338 (local address 127.1.244.254:42285)
0807 04:27:16.557334 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.557339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.557835 (+ 496us) server_negotiation.cc:407] Connection header received
0807 04:27:16.557971 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.557974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.558036 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.558113 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:16.558579 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.559226 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.560820 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.561201 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.561298 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.562272 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:16.562277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:16.562280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:16.562296 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:16.562325 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:16.562336 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:16.562438 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.562588 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.562628 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:27:16.566439 21477 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:27:16.573021 21477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:27:16.596369 21958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.584772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:44975 (local address 127.0.0.1:40942)
0807 04:27:16.588053 (+ 3281us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.588073 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.588220 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.589015 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.589021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.589057 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:16.589502 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.589517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.590890 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.590898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.593631 (+ 2733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.593648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.595295 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.595302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.595565 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.595621 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.595756 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.595967 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2931,"mutex_wait_us":3,"thread_start_us":2804,"threads_started":1}
I20260807 04:27:16.596371 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.584844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.1.244.194:44975)
0807 04:27:16.586283 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.586296 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.588183 (+ 1887us) server_negotiation.cc:407] Connection header received
0807 04:27:16.588405 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.588415 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.588668 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.588866 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:16.589721 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.590743 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.593864 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.595081 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.595223 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.595470 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.595828 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.596028 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1212,"thread_start_us":1039,"threads_started":1}
I20260807 04:27:16.599447 21961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.586947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39079 (local address 127.0.0.1:49906)
0807 04:27:16.588728 (+ 1781us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.588745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.588847 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.591949 (+ 3102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.591954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.591979 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:16.592502 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.592514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.594579 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.594586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.596721 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.596734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.598774 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.598780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.598966 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.599012 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.599114 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.599197 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1665,"mutex_wait_us":6,"thread_start_us":1545,"threads_started":1}
I20260807 04:27:16.599793 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.584748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:42231 (local address 127.0.0.1:59554)
0807 04:27:16.586974 (+ 2226us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.587025 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.587249 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.592243 (+ 4994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.592250 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.592280 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:16.592763 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.592775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.594851 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.594859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.597037 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.597053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.598974 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.598981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.599158 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.599188 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.599251 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.599375 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1920,"mutex_wait_us":7,"thread_start_us":1722,"threads_started":1}
I20260807 04:27:16.600325 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.585372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.1.244.193:42231)
0807 04:27:16.591663 (+ 6291us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.591676 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.591700 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:27:16.591781 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.591786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.592013 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.593005 (+ 992us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:16.593243 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.594708 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.597276 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.598718 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.599537 (+ 819us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.599745 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.599815 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.599991 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6154,"thread_start_us":6048,"threads_started":1}
I20260807 04:27:16.600401 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.587659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49906 (local address 127.1.244.195:39079)
0807 04:27:16.591370 (+ 3711us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.591378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.591426 (+ 48us) server_negotiation.cc:407] Connection header received
0807 04:27:16.591518 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.591525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.591739 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.591960 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:16.592779 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.594429 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.596972 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.598584 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.598743 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.598999 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.599794 (+ 795us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.599992 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3543,"thread_start_us":3377,"threads_started":1}
I20260807 04:27:16.600666 21740 tablet_service.cc:1467] Processing CreateTablet for tablet e3c25598bbbf488594d6ebb0ea898cc5 (DEFAULT_TABLE table=test-table [id=b4a5030ac7884941b071e5246de4c29d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:16.601991 21740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c25598bbbf488594d6ebb0ea898cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:16.603627 21604 tablet_service.cc:1467] Processing CreateTablet for tablet e3c25598bbbf488594d6ebb0ea898cc5 (DEFAULT_TABLE table=test-table [id=b4a5030ac7884941b071e5246de4c29d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:16.604938 21604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c25598bbbf488594d6ebb0ea898cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:16.610843 21875 tablet_service.cc:1467] Processing CreateTablet for tablet e3c25598bbbf488594d6ebb0ea898cc5 (DEFAULT_TABLE table=test-table [id=b4a5030ac7884941b071e5246de4c29d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:16.612041 21875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c25598bbbf488594d6ebb0ea898cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:16.615983 21963 tablet_bootstrap.cc:492] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Bootstrap starting.
I20260807 04:27:16.618110 21964 tablet_bootstrap.cc:492] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Bootstrap starting.
I20260807 04:27:16.620148 21963 tablet_bootstrap.cc:654] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:16.621068 21964 tablet_bootstrap.cc:654] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:16.621448 21963 log.cc:824] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:16.622151 21964 log.cc:824] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:16.624668 21963 tablet_bootstrap.cc:492] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: No bootstrap required, opened a new log
I20260807 04:27:16.624948 21963 ts_tablet_manager.cc:1397] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260807 04:27:16.626628 21967 tablet_bootstrap.cc:492] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Bootstrap starting.
I20260807 04:27:16.628238 21964 tablet_bootstrap.cc:492] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: No bootstrap required, opened a new log
I20260807 04:27:16.628707 21964 ts_tablet_manager.cc:1397] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 04:27:16.630100 21967 tablet_bootstrap.cc:654] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:16.631400 21967 log.cc:824] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:16.635455 21963 raft_consensus.cc:348] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.636240 21963 raft_consensus.cc:374] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:16.636413 21963 raft_consensus.cc:729] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66287ddd73b041f9aa02cccc1048aaca, State: Initialized, Role: FOLLOWER
I20260807 04:27:16.637209 21963 consensus_queue.cc:260] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.639350 21808 heartbeater.cc:503] Master 127.1.244.254:42285 was elected leader, sending a full tablet report...
I20260807 04:27:16.640573 21963 ts_tablet_manager.cc:1428] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
I20260807 04:27:16.641538 21964 raft_consensus.cc:348] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.641971 21964 raft_consensus.cc:374] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:16.642076 21964 raft_consensus.cc:729] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049b8ee501584aea85caaf4f75ea94f0, State: Initialized, Role: FOLLOWER
I20260807 04:27:16.642683 21967 tablet_bootstrap.cc:492] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: No bootstrap required, opened a new log
I20260807 04:27:16.642732 21964 consensus_queue.cc:260] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.642966 21967 ts_tablet_manager.cc:1397] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.012s
I20260807 04:27:16.645632 21670 heartbeater.cc:503] Master 127.1.244.254:42285 was elected leader, sending a full tablet report...
I20260807 04:27:16.646234 21964 ts_tablet_manager.cc:1428] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260807 04:27:16.652521 21967 raft_consensus.cc:348] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.652866 21967 raft_consensus.cc:374] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:16.652984 21967 raft_consensus.cc:729] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 946a0478b4ad4e5591af1b1b48622e86, State: Initialized, Role: FOLLOWER
I20260807 04:27:16.653561 21967 consensus_queue.cc:260] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.654561 21943 heartbeater.cc:503] Master 127.1.244.254:42285 was elected leader, sending a full tablet report...
I20260807 04:27:16.654979 21967 ts_tablet_manager.cc:1428] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260807 04:27:16.757292 21970 raft_consensus.cc:482] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:16.757589 21970 raft_consensus.cc:504] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.759287 21970 leader_election.cc:288] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66287ddd73b041f9aa02cccc1048aaca (127.1.244.194:44975), 946a0478b4ad4e5591af1b1b48622e86 (127.1.244.195:39079)
I20260807 04:27:16.766218 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.759749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:41853 (local address 127.1.244.194:44975)
0807 04:27:16.760003 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.760011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.760768 (+ 757us) server_negotiation.cc:407] Connection header received
0807 04:27:16.760931 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.760936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.761019 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.761117 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:16.761778 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.762513 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.764430 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.765202 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.765324 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.765421 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.765807 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.765950 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 04:27:16.766238 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.760008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:47399 (local address 127.1.244.195:39079)
0807 04:27:16.760233 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.760240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.760934 (+ 694us) server_negotiation.cc:407] Connection header received
0807 04:27:16.761110 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.761116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.761187 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.761293 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:16.761919 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.762642 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.764521 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.765295 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.765483 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.765552 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.765875 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.766015 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 04:27:16.766408 21972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.759714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:44975 (local address 127.1.244.193:41853)
0807 04:27:16.760616 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.760636 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.760790 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.761145 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.761149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.761175 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:16.761623 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.761637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.762653 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.762658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.764254 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.764266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.765417 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.765424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.765590 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.765657 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.765725 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.765873 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":20,"spinlock_wait_cycles":34048,"thread_start_us":462,"threads_started":1}
I20260807 04:27:16.766536 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.759866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:39079 (local address 127.1.244.193:47399)
0807 04:27:16.760792 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:16.760811 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:16.760966 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:16.761316 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:16.761319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:16.761360 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:16.761758 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.761770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.762781 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.762784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.764308 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:16.764320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.765502 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.765507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:16.765665 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.765697 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 04:27:16.765829 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 04:27:16.765948 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":38,"thread_start_us":570,"threads_started":1}
I20260807 04:27:16.767447 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c25598bbbf488594d6ebb0ea898cc5" candidate_uuid: "049b8ee501584aea85caaf4f75ea94f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66287ddd73b041f9aa02cccc1048aaca" is_pre_election: true
I20260807 04:27:16.767493 21895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c25598bbbf488594d6ebb0ea898cc5" candidate_uuid: "049b8ee501584aea85caaf4f75ea94f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946a0478b4ad4e5591af1b1b48622e86" is_pre_election: true
I20260807 04:27:16.768021 21761 raft_consensus.cc:2454] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 049b8ee501584aea85caaf4f75ea94f0 in term 0.
I20260807 04:27:16.768100 21895 raft_consensus.cc:2454] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 049b8ee501584aea85caaf4f75ea94f0 in term 0.
I20260807 04:27:16.768725 21554 leader_election.cc:302] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 049b8ee501584aea85caaf4f75ea94f0, 66287ddd73b041f9aa02cccc1048aaca, 946a0478b4ad4e5591af1b1b48622e86; no voters:
I20260807 04:27:16.769120 21970 raft_consensus.cc:2781] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:27:16.769299 21970 raft_consensus.cc:482] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:27:16.769402 21970 raft_consensus.cc:3037] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:16.771585 21970 raft_consensus.cc:504] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
I20260807 04:27:16.772305 21970 leader_election.cc:288] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [CANDIDATE]: Term 1 election: Requested vote from peers 66287ddd73b041f9aa02cccc1048aaca (127.1.244.194:44975), 946a0478b4ad4e5591af1b1b48622e86 (127.1.244.195:39079)
I20260807 04:27:16.772713 21761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c25598bbbf488594d6ebb0ea898cc5" candidate_uuid: "049b8ee501584aea85caaf4f75ea94f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66287ddd73b041f9aa02cccc1048aaca"
I20260807 04:27:16.772740 21895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c25598bbbf488594d6ebb0ea898cc5" candidate_uuid: "049b8ee501584aea85caaf4f75ea94f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "946a0478b4ad4e5591af1b1b48622e86"
I20260807 04:27:16.772938 21895 raft_consensus.cc:3037] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:16.772927 21761 raft_consensus.cc:3037] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:16.776258 21895 raft_consensus.cc:2454] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 049b8ee501584aea85caaf4f75ea94f0 in term 1.
I20260807 04:27:16.776258 21761 raft_consensus.cc:2454] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 049b8ee501584aea85caaf4f75ea94f0 in term 1.
I20260807 04:27:16.776640 21556 leader_election.cc:302] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049b8ee501584aea85caaf4f75ea94f0, 946a0478b4ad4e5591af1b1b48622e86; no voters:
I20260807 04:27:16.776919 21970 raft_consensus.cc:2781] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:16.777361 21970 raft_consensus.cc:686] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 1 LEADER]: Becoming Leader. State: Replica: 049b8ee501584aea85caaf4f75ea94f0, State: Running, Role: LEADER
I20260807 04:27:16.777808 21970 consensus_queue.cc:237] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } }
W20260807 04:27:16.782636 21944 tablet.cc:2369] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:16.782564 21477 catalog_manager.cc:5495] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 049b8ee501584aea85caaf4f75ea94f0 (127.1.244.193). New cstate: current_term: 1 leader_uuid: "049b8ee501584aea85caaf4f75ea94f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "049b8ee501584aea85caaf4f75ea94f0" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 42231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 } health_report { overall_health: UNKNOWN } } }
W20260807 04:27:16.830425 21809 tablet.cc:2369] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:16.834096 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.828525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.1.244.193:42231)
0807 04:27:16.828733 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.828743 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.828910 (+ 167us) server_negotiation.cc:407] Connection header received
0807 04:27:16.829073 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.829077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.829162 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.829283 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:27:16.829738 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.830409 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.832106 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.832521 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.832613 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.833045 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:27:16.833499 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:27:16.833619 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.833743 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.833778 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:27:16.842293 21895 raft_consensus.cc:1261] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 1 FOLLOWER]: Refusing update from remote peer 049b8ee501584aea85caaf4f75ea94f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:16.842298 21761 raft_consensus.cc:1261] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 1 FOLLOWER]: Refusing update from remote peer 049b8ee501584aea85caaf4f75ea94f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:16.843340 21975 consensus_queue.cc:1035] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "946a0478b4ad4e5591af1b1b48622e86" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 39079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:16.843670 21970 consensus_queue.cc:1035] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66287ddd73b041f9aa02cccc1048aaca" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:16.855010 21979 mvcc.cc:205] Tried to move back new op lower bound from 7315770723695587328 to 7315770723445788672. Current Snapshot: MvccSnapshot[applied={T|T < 7315770723695587328}]
I20260807 04:27:16.856015 21978 mvcc.cc:205] Tried to move back new op lower bound from 7315770723695587328 to 7315770723445788672. Current Snapshot: MvccSnapshot[applied={T|T < 7315770723695587328}]
I20260807 04:27:16.868623 21477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:16.868888 21477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:16.872457 21477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 319bb6605753467291f4a398dbaa678d: Sending DeleteTablet for 3 replicas of tablet e3c25598bbbf488594d6ebb0ea898cc5
I20260807 04:27:16.873468 21740 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c25598bbbf488594d6ebb0ea898cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:16 UTC) from {username='slave'} at 127.0.0.1:40942
I20260807 04:27:16.873613 21604 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c25598bbbf488594d6ebb0ea898cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:16 UTC) from {username='slave'} at 127.0.0.1:59554
I20260807 04:27:16.873980 21875 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c25598bbbf488594d6ebb0ea898cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:16 UTC) from {username='slave'} at 127.0.0.1:49906
I20260807 04:27:16.876362 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 21537
I20260807 04:27:16.887064 21989 tablet_replica.cc:330] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: stopping tablet replica
I20260807 04:27:16.887573 21989 raft_consensus.cc:2229] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:27:16.888283 21989 raft_consensus.cc:2258] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:16.890213 21989 ts_tablet_manager.cc:1905] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:16.897715 21989 ts_tablet_manager.cc:1918] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:16.897933 21989 log.cc:1196] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/e3c25598bbbf488594d6ebb0ea898cc5
I20260807 04:27:16.898514 21989 ts_tablet_manager.cc:1939] T e3c25598bbbf488594d6ebb0ea898cc5 P 049b8ee501584aea85caaf4f75ea94f0: Deleting consensus metadata
I20260807 04:27:16.900497 21463 catalog_manager.cc:4841] TS 049b8ee501584aea85caaf4f75ea94f0 (127.1.244.193:42231): tablet e3c25598bbbf488594d6ebb0ea898cc5 (table test-table [id=b4a5030ac7884941b071e5246de4c29d]) successfully deleted
I20260807 04:27:16.904702 21990 tablet_replica.cc:330] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: stopping tablet replica
I20260807 04:27:16.905318 21990 raft_consensus.cc:2229] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:16.906035 21990 raft_consensus.cc:2258] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:16.907001 21988 tablet_replica.cc:330] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: stopping tablet replica
I20260807 04:27:16.907449 21988 raft_consensus.cc:2229] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:16.908003 21988 raft_consensus.cc:2258] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:16.908812 21990 ts_tablet_manager.cc:1905] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:16.909777 21988 ts_tablet_manager.cc:1905] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:16.913376 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:16.880177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.1.244.193:42231)
0807 04:27:16.880969 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:16.880978 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:16.904061 (+ 23083us) server_negotiation.cc:407] Connection header received
0807 04:27:16.904408 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:16.904414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:16.904566 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:16.904659 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:16.905580 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.906975 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.909303 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:16.909967 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:16.910067 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:16.911600 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:16.911612 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:16.911629 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:16.912419 (+ 790us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:16.912475 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:16.912499 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:16.912574 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:16.912844 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 04:27:16.912954 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629}
I20260807 04:27:16.914537 21665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:16.916258 21990 ts_tablet_manager.cc:1918] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:16.916471 21990 log.cc:1196] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/e3c25598bbbf488594d6ebb0ea898cc5
I20260807 04:27:16.917269 21990 ts_tablet_manager.cc:1939] T e3c25598bbbf488594d6ebb0ea898cc5 P 946a0478b4ad4e5591af1b1b48622e86: Deleting consensus metadata
I20260807 04:27:16.917459 21988 ts_tablet_manager.cc:1918] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:16.917665 21988 log.cc:1196] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/e3c25598bbbf488594d6ebb0ea898cc5
I20260807 04:27:16.918222 21988 ts_tablet_manager.cc:1939] T e3c25598bbbf488594d6ebb0ea898cc5 P 66287ddd73b041f9aa02cccc1048aaca: Deleting consensus metadata
I20260807 04:27:16.919442 21464 catalog_manager.cc:4841] TS 946a0478b4ad4e5591af1b1b48622e86 (127.1.244.195:39079): tablet e3c25598bbbf488594d6ebb0ea898cc5 (table test-table [id=b4a5030ac7884941b071e5246de4c29d]) successfully deleted
I20260807 04:27:16.919870 21462 catalog_manager.cc:4841] TS 66287ddd73b041f9aa02cccc1048aaca (127.1.244.194:44975): tablet e3c25598bbbf488594d6ebb0ea898cc5 (table test-table [id=b4a5030ac7884941b071e5246de4c29d]) successfully deleted
I20260807 04:27:17.015053 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 21537
I20260807 04:27:17.036041 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 21674
I20260807 04:27:17.043100 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:17.036882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.1.244.194:44975)
0807 04:27:17.037066 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:17.037073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:17.037171 (+ 98us) server_negotiation.cc:407] Connection header received
0807 04:27:17.037408 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:17.037414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:17.037541 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:17.037646 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:17.038037 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:17.038752 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:17.040092 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:17.040496 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:17.040635 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:17.041737 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:17.041755 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:17.041771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:17.042330 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:17.042375 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:17.042394 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:17.042516 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:17.042652 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 04:27:17.042758 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:17.043838 21802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:17.133109 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 21674
I20260807 04:27:17.153632 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 21811
I20260807 04:27:17.160228 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:17.154464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49914 (local address 127.1.244.195:39079)
0807 04:27:17.154637 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:17.154642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:17.154660 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:27:17.154726 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:17.154730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:17.154826 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:17.154912 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:17.155273 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:17.156238 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:17.157502 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:17.157892 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:17.157986 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:17.158935 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:17.158943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:17.158958 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:17.159542 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:17.159589 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:17.159612 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:17.159699 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:17.159809 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 04:27:17.159962 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:17.160941 21936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:17.250831 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 21811
I20260807 04:27:17.269625 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 21445
I20260807 04:27:17.270735 21508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:17.379788 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 21445
2026-08-07T04:27:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2824 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:17 dist-test-slave-zkpd krb5kdc[21998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:17 dist-test-slave-zkpd krb5kdc[21998](info): set up 2 sockets
Aug 07 04:27:17 dist-test-slave-zkpd krb5kdc[21998](info): commencing operation
krb5kdc: starting...
W20260807 04:27:18.534767 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:18 dist-test-slave-zkpd krb5kdc[21998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:18.821012 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41705
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:19.150597 22012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:19.150874 22012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:19.151049 22012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:19.158113 22012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:19.158214 22012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:19.158260 22012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:19.158349 22012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:19.158416 22012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:19.168018 22012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:41705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:19.169299 22012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:19.170647 22012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:19.182574 22021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:19.182600 22019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:19.183194 22018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:19.183773 22012 server_base.cc:1034] running on GCE node
I20260807 04:27:19.184609 22012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:19.186077 22012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:19.187322 22012 hybrid_clock.cc:648] HybridClock initialized: now 1786076839187269 us; error 46 us; skew 500 ppm
I20260807 04:27:19.187696 22012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:19 dist-test-slave-zkpd krb5kdc[21998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076839, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:19.190933 22012 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:19.192579 22012 webserver.cc:460] Webserver started at http://127.1.244.254:45331/ using document root <none> and password file <none>
I20260807 04:27:19.193296 22012 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:19.193449 22012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:19.193779 22012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:19.196816 22012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "f8554254cbbf4bfb9558b63cef6ad9b9"
format_stamp: "Formatted at 2026-08-07 04:27:19 on dist-test-slave-zkpd"
server_key: "e7f6f8297cb22b246de8cbfd943c1597"
server_key_iv: "636ec557b425e56de8d3633f059d6d57"
server_key_version: "encryptionkey@0"
I20260807 04:27:19.197588 22012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "f8554254cbbf4bfb9558b63cef6ad9b9"
format_stamp: "Formatted at 2026-08-07 04:27:19 on dist-test-slave-zkpd"
server_key: "e7f6f8297cb22b246de8cbfd943c1597"
server_key_iv: "636ec557b425e56de8d3633f059d6d57"
server_key_version: "encryptionkey@0"
I20260807 04:27:19.202854 22012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 04:27:19.206420 22028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:19.207710 22012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:27:19.207945 22012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "f8554254cbbf4bfb9558b63cef6ad9b9"
format_stamp: "Formatted at 2026-08-07 04:27:19 on dist-test-slave-zkpd"
server_key: "e7f6f8297cb22b246de8cbfd943c1597"
server_key_iv: "636ec557b425e56de8d3633f059d6d57"
server_key_version: "encryptionkey@0"
I20260807 04:27:19.208156 22012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:19.242261 22012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:19.246452 22012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:19.246744 22012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:19.265363 22012 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:41705
I20260807 04:27:19.265410 22080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:41705 every 8 connection(s)
I20260807 04:27:19.266685 22012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:19.268103 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22012
I20260807 04:27:19.268314 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:19.268922 2003 external_mini_cluster.cc:1428] Setting key cddcd2035698010e47c2e1d7be163fbd
I20260807 04:27:19.271406 22081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:19.282397 22081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9: Bootstrap starting.
Aug 07 04:27:19 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:19.285738 22081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:19.286820 22081 log.cc:824] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:19.289731 22081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9: No bootstrap required, opened a new log
I20260807 04:27:19.291404 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:19.270837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.1.244.254:41705)
0807 04:27:19.271638 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:19.271650 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:19.271688 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:27:19.273372 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:19.273392 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:19.273796 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:19.274441 (+ 645us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:19.276290 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:19.277607 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:19.278883 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:19.279585 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:19.279801 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:19.284306 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:19.284314 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:19.284326 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:19.284361 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:19.287462 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:19.288253 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:19.288266 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:19.288272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:19.288442 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:19.288836 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:19.288846 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:19.288848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:19.289313 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:19.289629 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:19.290044 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 04:27:19.290484 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":323,"threads_started":1}
I20260807 04:27:19.297646 22081 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41705 } }
I20260807 04:27:19.298121 22081 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:19.298246 22081 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8554254cbbf4bfb9558b63cef6ad9b9, State: Initialized, Role: FOLLOWER
I20260807 04:27:19.298866 22081 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41705 } }
I20260807 04:27:19.299175 22081 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:19.299311 22081 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:19.299489 22081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:19.301666 22081 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41705 } }
I20260807 04:27:19.302141 22081 leader_election.cc:302] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8554254cbbf4bfb9558b63cef6ad9b9; no voters:
I20260807 04:27:19.302829 22081 leader_election.cc:288] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:19.302927 22086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:19.303768 22086 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [term 1 LEADER]: Becoming Leader. State: Replica: f8554254cbbf4bfb9558b63cef6ad9b9, State: Running, Role: LEADER
I20260807 04:27:19.304428 22086 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41705 } }
I20260807 04:27:19.305045 22081 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:19.307235 22087 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41705 } } }
I20260807 04:27:19.307730 22087 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:19.307552 22088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8554254cbbf4bfb9558b63cef6ad9b9. Latest consensus state: current_term: 1 leader_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8554254cbbf4bfb9558b63cef6ad9b9" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 41705 } } }
I20260807 04:27:19.307875 22088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:19.313395 22094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:19.317497 22094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:19.325260 22094 catalog_manager.cc:1347] Generated new cluster ID: b04b37975bd143f792c9b8f9d92afba9
I20260807 04:27:19.325376 22094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:19.357697 22094 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:19.359004 22094 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:19.366212 22094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9: Generated new TSK 0
I20260807 04:27:19.366959 22094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.244.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
Entry for principal kudu/127.1.244.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab.
I20260807 04:27:19.440196 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41705
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:19.782976 22109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:19.783284 22109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:19.783469 22109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:19.790737 22109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:19.790882 22109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:19.791064 22109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:27:19.801018 22109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.193.keytab
--principal=kudu/127.1.244.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:19.802244 22109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:19.803557 22109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:19.816908 22115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:19.816934 22114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:19.817185 22117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:19.817770 22109 server_base.cc:1034] running on GCE node
I20260807 04:27:19.818352 22109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:19.819231 22109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:19.820454 22109 hybrid_clock.cc:648] HybridClock initialized: now 1786076839820414 us; error 49 us; skew 500 ppm
I20260807 04:27:19.820775 22109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:19 dist-test-slave-zkpd krb5kdc[21998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076839, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:19.823797 22109 init.cc:377] Logged in from keytab as kudu/127.1.244.193@KRBTEST.COM (short username kudu)
I20260807 04:27:19.825237 22109 webserver.cc:460] Webserver started at http://127.1.244.193:42719/ using document root <none> and password file <none>
I20260807 04:27:19.825875 22109 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:19.826025 22109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:19.826329 22109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:19.829203 22109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc"
format_stamp: "Formatted at 2026-08-07 04:27:19 on dist-test-slave-zkpd"
server_key: "021d4d7e71426c55d1f5a5c49011eb2f"
server_key_iv: "2221f0ad0787037afc99d2dead144c5e"
server_key_version: "encryptionkey@0"
I20260807 04:27:19.829983 22109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc"
format_stamp: "Formatted at 2026-08-07 04:27:19 on dist-test-slave-zkpd"
server_key: "021d4d7e71426c55d1f5a5c49011eb2f"
server_key_iv: "2221f0ad0787037afc99d2dead144c5e"
server_key_version: "encryptionkey@0"
I20260807 04:27:19.834939 22109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:27:19.838222 22125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:19.839440 22109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:27:19.839614 22109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc"
format_stamp: "Formatted at 2026-08-07 04:27:19 on dist-test-slave-zkpd"
server_key: "021d4d7e71426c55d1f5a5c49011eb2f"
server_key_iv: "2221f0ad0787037afc99d2dead144c5e"
server_key_version: "encryptionkey@0"
I20260807 04:27:19.839807 22109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:19.860601 22109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:19.864377 22109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:19.864655 22109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:19.865605 22109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:19.867029 22109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:19.867134 22109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:19.867275 22109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:19.867348 22109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:19.900844 22109 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:34717
I20260807 04:27:19.900935 22240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:34717 every 8 connection(s)
I20260807 04:27:19.902150 22109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:27:19.905534 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22109
I20260807 04:27:19.905761 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:27:19.906299 2003 external_mini_cluster.cc:1428] Setting key 283767545b68467ffbdf8feeba3bc105
Aug 07 04:27:19 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076839, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.193@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:19.923764 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:19.904678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:51033 (local address 127.1.244.254:41705)
0807 04:27:19.905136 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:19.905141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:19.906338 (+ 1197us) server_negotiation.cc:407] Connection header received
0807 04:27:19.908412 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:19.908418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:19.908504 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:19.908614 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:19.910321 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:19.911071 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:19.912822 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:19.913233 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:19.913377 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:19.916782 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:19.916791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:19.916798 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:19.916844 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:19.920443 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:19.921392 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:19.921395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:19.921398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:19.921514 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:19.922059 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:19.922062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:19.922064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:19.922312 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:19.922477 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:19.923176 (+ 699us) server_negotiation.cc:299] Negotiation successful
0807 04:27:19.923397 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":188,"threads_started":1}
I20260807 04:27:19.924674 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:19.905288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41705 (local address 127.1.244.193:51033)
0807 04:27:19.906189 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:19.906233 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:19.908122 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:19.908838 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:19.908846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:19.909502 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:19.910134 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:19.910164 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:19.911232 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:19.911237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:19.912685 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:19.912698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:19.913478 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:19.913485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:19.913713 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:19.914399 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:19.914421 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:19.916544 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:19.920595 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:19.920606 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:19.920623 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:19.921270 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:19.921620 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:19.921626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:19.921630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:19.921939 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:19.922461 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:19.922469 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:19.922780 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 04:27:19.923044 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 04:27:19.923559 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":374,"threads_started":1}
I20260807 04:27:19.926256 22241 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41705
I20260807 04:27:19.926563 22241 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:19.927258 22241 heartbeater.cc:511] Master 127.1.244.254:41705 requested a full tablet report, sending...
I20260807 04:27:19.929410 22045 ts_manager.cc:194] Registered new tserver with Master: 26df49ec36c745e2ab34f2bcaa0ca6fc (127.1.244.193:34717)
I20260807 04:27:19.930792 22045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.193@KRBTEST.COM'} at 127.1.244.193:51033
WARNING: no policy specified for kudu/127.1.244.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
Entry for principal kudu/127.1.244.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab.
I20260807 04:27:19.970885 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41705
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:20.316711 22247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:20.316991 22247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:20.317200 22247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:20.324816 22247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:20.324934 22247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:20.325109 22247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:27:20.335516 22247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.194.keytab
--principal=kudu/127.1.244.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:20.336861 22247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:20.338284 22247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:20.351192 22254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:20.351271 22256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:20.351316 22253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:20.352130 22247 server_base.cc:1034] running on GCE node
I20260807 04:27:20.352746 22247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:20.353632 22247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:20.354849 22247 hybrid_clock.cc:648] HybridClock initialized: now 1786076840354803 us; error 37 us; skew 500 ppm
I20260807 04:27:20.355149 22247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:20 dist-test-slave-zkpd krb5kdc[21998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076840, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:20.363029 22247 init.cc:377] Logged in from keytab as kudu/127.1.244.194@KRBTEST.COM (short username kudu)
I20260807 04:27:20.364524 22247 webserver.cc:460] Webserver started at http://127.1.244.194:34909/ using document root <none> and password file <none>
I20260807 04:27:20.365195 22247 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:20.365304 22247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:20.365557 22247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:20.368454 22247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "0078ef84a22248e89bf1182639f15726"
format_stamp: "Formatted at 2026-08-07 04:27:20 on dist-test-slave-zkpd"
server_key: "37878f4d317eb51d4be654ccc7d6c9d7"
server_key_iv: "505752a63dc33527df75f5200ed7572e"
server_key_version: "encryptionkey@0"
I20260807 04:27:20.369252 22247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "0078ef84a22248e89bf1182639f15726"
format_stamp: "Formatted at 2026-08-07 04:27:20 on dist-test-slave-zkpd"
server_key: "37878f4d317eb51d4be654ccc7d6c9d7"
server_key_iv: "505752a63dc33527df75f5200ed7572e"
server_key_version: "encryptionkey@0"
I20260807 04:27:20.374220 22247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:20.377645 22263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:20.378913 22247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:27:20.379086 22247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "0078ef84a22248e89bf1182639f15726"
format_stamp: "Formatted at 2026-08-07 04:27:20 on dist-test-slave-zkpd"
server_key: "37878f4d317eb51d4be654ccc7d6c9d7"
server_key_iv: "505752a63dc33527df75f5200ed7572e"
server_key_version: "encryptionkey@0"
I20260807 04:27:20.379285 22247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:20.418620 22247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:20.423203 22247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:20.423511 22247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:20.424598 22247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:20.425940 22247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:20.426010 22247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:20.426146 22247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:20.426199 22247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:20.460196 22247 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:44449
I20260807 04:27:20.460258 22383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:44449 every 8 connection(s)
I20260807 04:27:20.461498 22247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:27:20.468947 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22247
I20260807 04:27:20.469148 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:27:20.469583 2003 external_mini_cluster.cc:1428] Setting key 1dada5671b549f3761cc7ee6edfce3fd
Aug 07 04:27:20 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076840, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.194@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:20.482241 22386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:20.464075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:44717 (local address 127.1.244.254:41705)
0807 04:27:20.464553 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:20.464558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:20.465332 (+ 774us) server_negotiation.cc:407] Connection header received
0807 04:27:20.467684 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:20.467691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:20.467783 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:20.467901 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:20.469502 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:20.470438 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:20.472198 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:20.472652 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:20.472785 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:20.476524 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:20.476529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:20.476532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:20.476563 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:20.479171 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:20.479931 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:20.479942 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:20.479946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:20.480075 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:20.480522 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:20.480526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:20.480528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:20.480781 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:20.480949 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:20.481607 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 04:27:20.481857 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
I20260807 04:27:20.483263 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:20.464509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41705 (local address 127.1.244.194:44717)
0807 04:27:20.465172 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:20.465209 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:20.467409 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:20.468163 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:20.468171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:20.468686 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:20.469334 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:20.469354 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:20.470608 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:20.470613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:20.472057 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:20.472069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:20.472892 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:20.472900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:20.473155 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:20.473883 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:20.473911 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:20.476286 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:20.479319 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:20.479327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:20.479343 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:20.479781 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:20.480189 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:20.480194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:20.480198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:20.480388 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:20.480916 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:20.480924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:20.481234 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 04:27:20.481510 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 04:27:20.482046 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":267,"threads_started":1}
I20260807 04:27:20.485070 22384 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41705
I20260807 04:27:20.485373 22384 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:20.486107 22384 heartbeater.cc:511] Master 127.1.244.254:41705 requested a full tablet report, sending...
I20260807 04:27:20.487533 22045 ts_manager.cc:194] Registered new tserver with Master: 0078ef84a22248e89bf1182639f15726 (127.1.244.194:44449)
I20260807 04:27:20.488399 22045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.194@KRBTEST.COM'} at 127.1.244.194:44717
WARNING: no policy specified for kudu/127.1.244.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
Entry for principal kudu/127.1.244.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab.
I20260807 04:27:20.536019 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41705
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:20.870347 22390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:20.870597 22390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:20.870750 22390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:20.877856 22390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:20.877954 22390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:20.878059 22390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:27:20.887486 22390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.195.keytab
--principal=kudu/127.1.244.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:41705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:20.888718 22390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:20.889982 22390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:20.903236 22395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:20.903265 22396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:20.903707 22399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:20.904263 22390 server_base.cc:1034] running on GCE node
I20260807 04:27:20.904861 22390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:20.905886 22390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:20.907127 22390 hybrid_clock.cc:648] HybridClock initialized: now 1786076840907075 us; error 50 us; skew 500 ppm
I20260807 04:27:20.907387 22390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:20 dist-test-slave-zkpd krb5kdc[21998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076840, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:20.910504 22390 init.cc:377] Logged in from keytab as kudu/127.1.244.195@KRBTEST.COM (short username kudu)
I20260807 04:27:20.911999 22390 webserver.cc:460] Webserver started at http://127.1.244.195:41025/ using document root <none> and password file <none>
I20260807 04:27:20.912631 22390 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:20.912730 22390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:20.913049 22390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:20.915859 22390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "db0e8e103a1541e4845536afab19e1b8"
format_stamp: "Formatted at 2026-08-07 04:27:20 on dist-test-slave-zkpd"
server_key: "738ede925e37aa248ea4e0ac4f71067c"
server_key_iv: "2c191240750b000d91435be28bd0b081"
server_key_version: "encryptionkey@0"
I20260807 04:27:20.916666 22390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "db0e8e103a1541e4845536afab19e1b8"
format_stamp: "Formatted at 2026-08-07 04:27:20 on dist-test-slave-zkpd"
server_key: "738ede925e37aa248ea4e0ac4f71067c"
server_key_iv: "2c191240750b000d91435be28bd0b081"
server_key_version: "encryptionkey@0"
I20260807 04:27:20.921505 22390 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:20.924810 22406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:20.926071 22390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:20.926200 22390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "db0e8e103a1541e4845536afab19e1b8"
format_stamp: "Formatted at 2026-08-07 04:27:20 on dist-test-slave-zkpd"
server_key: "738ede925e37aa248ea4e0ac4f71067c"
server_key_iv: "2c191240750b000d91435be28bd0b081"
server_key_version: "encryptionkey@0"
I20260807 04:27:20.926353 22390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:20.933576 22241 heartbeater.cc:503] Master 127.1.244.254:41705 was elected leader, sending a full tablet report...
I20260807 04:27:20.947564 22390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:20.951345 22390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:20.951634 22390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:20.952651 22390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:20.953948 22390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:20.954025 22390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:20.954202 22390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:20.954254 22390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:20.987761 22390 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:34849
I20260807 04:27:20.987876 22522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:34849 every 8 connection(s)
I20260807 04:27:20.989130 22390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:27:20.995335 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22390
I20260807 04:27:20.995534 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:27:20.996147 2003 external_mini_cluster.cc:1428] Setting key 59a4f4b8741d800ea48eca86655b2c56
Aug 07 04:27:20 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076840, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.195@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:21.008978 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:20.991623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59095 (local address 127.1.244.254:41705)
0807 04:27:20.992177 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:20.992184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:20.993071 (+ 887us) server_negotiation.cc:407] Connection header received
0807 04:27:20.994852 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:20.994857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:20.994930 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:20.995025 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:20.997072 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:20.998194 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:20.999947 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.000319 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.000423 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.003658 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:21.003663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:21.003666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:21.003696 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:21.006364 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.007089 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.007092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.007094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.007205 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.007554 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.007557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.007559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.007810 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:21.007991 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.008558 (+ 567us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.008711 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":272,"threads_started":1}
I20260807 04:27:21.010152 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:20.992192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:41705 (local address 127.1.244.195:59095)
0807 04:27:20.992907 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:20.992951 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:20.994588 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:20.995307 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:20.995321 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:20.996101 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:20.996856 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:20.996891 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:20.998356 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:20.998361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:20.999769 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:20.999782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.000501 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.000505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.000682 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.001269 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:27:21.001289 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:21.003466 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:21.006511 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:21.006518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:21.006577 (+ 59us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:21.006994 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:21.007294 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:27:21.007297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:27:21.007299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:27:21.007466 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:27:21.007967 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:21.007972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:21.008256 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 04:27:21.008473 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 04:27:21.008815 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":265,"threads_started":1}
I20260807 04:27:21.011543 22523 heartbeater.cc:347] Connected to a master server at 127.1.244.254:41705
I20260807 04:27:21.011884 22523 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:21.012612 22523 heartbeater.cc:511] Master 127.1.244.254:41705 requested a full tablet report, sending...
I20260807 04:27:21.014175 22044 ts_manager.cc:194] Registered new tserver with Master: db0e8e103a1541e4845536afab19e1b8 (127.1.244.195:34849)
I20260807 04:27:21.015091 22044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.244.195@KRBTEST.COM'} at 127.1.244.195:59095
I20260807 04:27:21.023079 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:27:21.039475 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.027276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.1.244.254:41705)
0807 04:27:21.027427 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.027432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.027843 (+ 411us) server_negotiation.cc:407] Connection header received
0807 04:27:21.028010 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.028014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.028071 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.028147 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:21.029603 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.030326 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.031805 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.032249 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.032343 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.034561 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:21.034566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:21.034569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:21.034598 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:21.037212 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.037778 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.037782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.037784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.037894 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.038212 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.038215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.038217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.038522 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:21.038700 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.038961 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.039124 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 04:27:21.043381 22044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:27:21.050148 22044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:27:21.078395 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.061774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:44449 (local address 127.0.0.1:57430)
0807 04:27:21.064066 (+ 2292us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:21.064088 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:21.064244 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:21.066723 (+ 2479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:21.066734 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:21.066774 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:21.067298 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.067314 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.071197 (+ 3883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.071206 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.075547 (+ 4341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.075565 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.077410 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.077416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.077621 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.077671 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 04:27:21.077796 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 04:27:21.077990 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2157,"thread_start_us":2314,"threads_started":1}
I20260807 04:27:21.079432 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.062344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.1.244.194:44449)
0807 04:27:21.063593 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.063603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.064402 (+ 799us) server_negotiation.cc:407] Connection header received
0807 04:27:21.064533 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.064544 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.064869 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.065086 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:21.067554 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.070990 (+ 3436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.075746 (+ 4756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.077227 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.077358 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.077592 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.077867 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.078051 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1080,"thread_start_us":1530,"threads_started":1}
I20260807 04:27:21.083024 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.063748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.1.244.195:34849)
0807 04:27:21.075586 (+ 11838us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.075597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.075632 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:27:21.075721 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.075731 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.075984 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.076168 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:21.076790 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.078136 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.080528 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.081878 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.082011 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.082233 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.082515 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.082672 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11631,"thread_start_us":11401,"threads_started":1}
I20260807 04:27:21.083067 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.063579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:34849 (local address 127.0.0.1:55868)
0807 04:27:21.074516 (+ 10937us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:21.074539 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:21.074678 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:21.076157 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:21.076161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:21.076186 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:21.076595 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.076606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.078293 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.078303 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.080309 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.080323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.082087 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.082093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.082267 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.082305 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 04:27:21.082423 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 04:27:21.082578 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10744,"mutex_wait_us":1,"thread_start_us":11188,"threads_started":1}
I20260807 04:27:21.083447 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.061562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34717 (local address 127.0.0.1:51456)
0807 04:27:21.062243 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:21.062291 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:21.062465 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:21.076410 (+ 13945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:21.076415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:21.076447 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:21.076877 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.076890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.078926 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.078931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.080867 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.080878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.082416 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.082423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.082627 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.082658 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 04:27:21.082753 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 04:27:21.082872 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":33,"thread_start_us":332,"threads_started":1}
I20260807 04:27:21.083813 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.061925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.1.244.193:34717)
0807 04:27:21.075835 (+ 13910us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.075843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.075867 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:27:21.075983 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.075990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.076187 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.077087 (+ 900us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:21.077288 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.078766 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.081011 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.081911 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.082071 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.082340 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.083327 (+ 987us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.083510 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13739,"thread_start_us":13587,"threads_started":1}
I20260807 04:27:21.083411 22315 tablet_service.cc:1467] Processing CreateTablet for tablet bba68c6d8e45461093185af96c7cfd71 (DEFAULT_TABLE table=test-table [id=81e32b3eb5e541f6ba42c0d88d12e7f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:21.085347 22315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba68c6d8e45461093185af96c7cfd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:21.086685 22457 tablet_service.cc:1467] Processing CreateTablet for tablet bba68c6d8e45461093185af96c7cfd71 (DEFAULT_TABLE table=test-table [id=81e32b3eb5e541f6ba42c0d88d12e7f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:21.087766 22457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba68c6d8e45461093185af96c7cfd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:21.087309 22174 tablet_service.cc:1467] Processing CreateTablet for tablet bba68c6d8e45461093185af96c7cfd71 (DEFAULT_TABLE table=test-table [id=81e32b3eb5e541f6ba42c0d88d12e7f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:21.088661 22174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba68c6d8e45461093185af96c7cfd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:21.098459 22543 tablet_bootstrap.cc:492] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Bootstrap starting.
I20260807 04:27:21.100471 22544 tablet_bootstrap.cc:492] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Bootstrap starting.
I20260807 04:27:21.102872 22545 tablet_bootstrap.cc:492] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Bootstrap starting.
I20260807 04:27:21.103528 22543 tablet_bootstrap.cc:654] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:21.104094 22544 tablet_bootstrap.cc:654] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:21.104867 22543 log.cc:824] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:21.105274 22544 log.cc:824] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:21.105888 22545 tablet_bootstrap.cc:654] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:21.106993 22545 log.cc:824] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:21.108330 22544 tablet_bootstrap.cc:492] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: No bootstrap required, opened a new log
I20260807 04:27:21.108340 22543 tablet_bootstrap.cc:492] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: No bootstrap required, opened a new log
I20260807 04:27:21.108594 22543 ts_tablet_manager.cc:1397] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 04:27:21.108594 22544 ts_tablet_manager.cc:1397] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260807 04:27:21.109583 22545 tablet_bootstrap.cc:492] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: No bootstrap required, opened a new log
I20260807 04:27:21.109836 22545 ts_tablet_manager.cc:1397] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260807 04:27:21.117343 22545 raft_consensus.cc:348] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.117707 22545 raft_consensus.cc:374] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:21.117820 22545 raft_consensus.cc:729] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26df49ec36c745e2ab34f2bcaa0ca6fc, State: Initialized, Role: FOLLOWER
I20260807 04:27:21.118429 22545 consensus_queue.cc:260] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.119310 22543 raft_consensus.cc:348] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.119310 22544 raft_consensus.cc:348] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.119722 22543 raft_consensus.cc:374] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:21.119722 22544 raft_consensus.cc:374] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:21.119813 22543 raft_consensus.cc:729] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0e8e103a1541e4845536afab19e1b8, State: Initialized, Role: FOLLOWER
I20260807 04:27:21.119813 22544 raft_consensus.cc:729] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0078ef84a22248e89bf1182639f15726, State: Initialized, Role: FOLLOWER
I20260807 04:27:21.120175 22545 ts_tablet_manager.cc:1428] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260807 04:27:21.120434 22543 consensus_queue.cc:260] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.120500 22544 consensus_queue.cc:260] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.121976 22523 heartbeater.cc:503] Master 127.1.244.254:41705 was elected leader, sending a full tablet report...
I20260807 04:27:21.122164 22544 ts_tablet_manager.cc:1428] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 04:27:21.122604 22543 ts_tablet_manager.cc:1428] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260807 04:27:21.122759 22384 heartbeater.cc:503] Master 127.1.244.254:41705 was elected leader, sending a full tablet report...
W20260807 04:27:21.154537 22242 tablet.cc:2369] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:27:21.213644 22385 tablet.cc:2369] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 04:27:21.240816 22524 tablet.cc:2369] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:21.424845 22550 raft_consensus.cc:482] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:21.425153 22550 raft_consensus.cc:504] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.426877 22550 leader_election.cc:288] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26df49ec36c745e2ab34f2bcaa0ca6fc (127.1.244.193:34717), 0078ef84a22248e89bf1182639f15726 (127.1.244.194:44449)
I20260807 04:27:21.434185 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.427313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:39553 (local address 127.1.244.194:44449)
0807 04:27:21.427534 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.427542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.428115 (+ 573us) server_negotiation.cc:407] Connection header received
0807 04:27:21.428264 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.428267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.428324 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.428394 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:21.428950 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.429710 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.432529 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.433313 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.433443 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.433496 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.433779 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.433921 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:27:21.434290 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.427367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:44449 (local address 127.1.244.195:39553)
0807 04:27:21.428007 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:21.428025 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:21.428148 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:21.428437 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:21.428440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:21.428460 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:21.428769 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.428779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.429925 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.429930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.432380 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.432395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.433475 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.433480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.433593 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.433618 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 04:27:21.433745 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 04:27:21.433883 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":148,"spinlock_wait_cycles":1920,"thread_start_us":330,"threads_started":1}
I20260807 04:27:21.435254 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.427218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:34717 (local address 127.1.244.195:52551)
0807 04:27:21.427623 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:21.427651 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:21.427830 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:21.428535 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:21.428538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:21.428575 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:21.428802 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.428814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.429958 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.429965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.432380 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:21.432395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.434605 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.434610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:21.434750 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.434773 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 04:27:21.434892 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 04:27:21.435029 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":123}
I20260807 04:27:21.435268 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.427381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:52551 (local address 127.1.244.193:34717)
0807 04:27:21.427602 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.427610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.427972 (+ 362us) server_negotiation.cc:407] Connection header received
0807 04:27:21.428064 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.428071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.428154 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.428236 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:21.428963 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.429832 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.433573 (+ 3741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.434470 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.434618 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.434692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.434927 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.435027 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:27:21.435717 22336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba68c6d8e45461093185af96c7cfd71" candidate_uuid: "db0e8e103a1541e4845536afab19e1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0078ef84a22248e89bf1182639f15726" is_pre_election: true
I20260807 04:27:21.436367 22336 raft_consensus.cc:2454] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db0e8e103a1541e4845536afab19e1b8 in term 0.
I20260807 04:27:21.436496 22194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba68c6d8e45461093185af96c7cfd71" candidate_uuid: "db0e8e103a1541e4845536afab19e1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" is_pre_election: true
I20260807 04:27:21.437058 22194 raft_consensus.cc:2454] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db0e8e103a1541e4845536afab19e1b8 in term 0.
I20260807 04:27:21.437103 22407 leader_election.cc:302] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0078ef84a22248e89bf1182639f15726, db0e8e103a1541e4845536afab19e1b8; no voters:
I20260807 04:27:21.437528 22550 raft_consensus.cc:2781] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:27:21.437721 22550 raft_consensus.cc:482] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:27:21.437839 22550 raft_consensus.cc:3037] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:21.440083 22550 raft_consensus.cc:504] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.440652 22550 leader_election.cc:288] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [CANDIDATE]: Term 1 election: Requested vote from peers 26df49ec36c745e2ab34f2bcaa0ca6fc (127.1.244.193:34717), 0078ef84a22248e89bf1182639f15726 (127.1.244.194:44449)
I20260807 04:27:21.441092 22194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba68c6d8e45461093185af96c7cfd71" candidate_uuid: "db0e8e103a1541e4845536afab19e1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc"
I20260807 04:27:21.441131 22336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba68c6d8e45461093185af96c7cfd71" candidate_uuid: "db0e8e103a1541e4845536afab19e1b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0078ef84a22248e89bf1182639f15726"
I20260807 04:27:21.441291 22194 raft_consensus.cc:3037] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:21.441323 22336 raft_consensus.cc:3037] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:21.443601 22194 raft_consensus.cc:2454] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db0e8e103a1541e4845536afab19e1b8 in term 1.
I20260807 04:27:21.443594 22336 raft_consensus.cc:2454] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db0e8e103a1541e4845536afab19e1b8 in term 1.
I20260807 04:27:21.444041 22407 leader_election.cc:302] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0078ef84a22248e89bf1182639f15726, db0e8e103a1541e4845536afab19e1b8; no voters:
I20260807 04:27:21.444326 22550 raft_consensus.cc:2781] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:21.444761 22550 raft_consensus.cc:686] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 1 LEADER]: Becoming Leader. State: Replica: db0e8e103a1541e4845536afab19e1b8, State: Running, Role: LEADER
I20260807 04:27:21.445291 22550 consensus_queue.cc:237] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } }
I20260807 04:27:21.449815 22044 catalog_manager.cc:5495] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to db0e8e103a1541e4845536afab19e1b8 (127.1.244.195). New cstate: current_term: 1 leader_uuid: "db0e8e103a1541e4845536afab19e1b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db0e8e103a1541e4845536afab19e1b8" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 34849 } health_report { overall_health: HEALTHY } } }
I20260807 04:27:21.524960 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.519630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55874 (local address 127.1.244.195:34849)
0807 04:27:21.519864 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.519870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.519914 (+ 44us) server_negotiation.cc:407] Connection header received
0807 04:27:21.520033 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.520037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.520113 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.520268 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:27:21.520638 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.521355 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.522763 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.523161 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.523297 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.523827 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:27:21.524390 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:27:21.524555 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.524618 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.524699 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:27:21.533025 22336 raft_consensus.cc:1261] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 1 FOLLOWER]: Refusing update from remote peer db0e8e103a1541e4845536afab19e1b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:21.533021 22194 raft_consensus.cc:1261] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 1 FOLLOWER]: Refusing update from remote peer db0e8e103a1541e4845536afab19e1b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:21.534034 22553 consensus_queue.cc:1035] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0078ef84a22248e89bf1182639f15726" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:21.534365 22550 consensus_queue.cc:1035] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26df49ec36c745e2ab34f2bcaa0ca6fc" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:21.543148 22556 mvcc.cc:205] Tried to move back new op lower bound from 7315770742909345792 to 7315770742564212736. Current Snapshot: MvccSnapshot[applied={T|T < 7315770742909345792}]
I20260807 04:27:21.544627 22558 mvcc.cc:205] Tried to move back new op lower bound from 7315770742909345792 to 7315770742564212736. Current Snapshot: MvccSnapshot[applied={T|T < 7315770742909345792}]
I20260807 04:27:21.557808 22043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:21.558215 22043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:21.562675 22043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8554254cbbf4bfb9558b63cef6ad9b9: Sending DeleteTablet for 3 replicas of tablet bba68c6d8e45461093185af96c7cfd71
I20260807 04:27:21.563745 22315 tablet_service.cc:1514] Processing DeleteTablet for tablet bba68c6d8e45461093185af96c7cfd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:21 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:57430
I20260807 04:27:21.563773 22174 tablet_service.cc:1514] Processing DeleteTablet for tablet bba68c6d8e45461093185af96c7cfd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:21 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:51456
I20260807 04:27:21.563854 22173 tablet_service.cc:1514] Processing DeleteTablet for tablet bba68c6d8e45461093185af96c7cfd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:21 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:51456
I20260807 04:27:21.564939 22567 tablet_replica.cc:330] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: stopping tablet replica
I20260807 04:27:21.565389 22457 tablet_service.cc:1514] Processing DeleteTablet for tablet bba68c6d8e45461093185af96c7cfd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:21 UTC) from {username='kudu', principal='kudu/127.1.244.254@KRBTEST.COM'} at 127.0.0.1:55868
I20260807 04:27:21.565634 22567 raft_consensus.cc:2229] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:21.566267 22567 raft_consensus.cc:2258] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:21.567508 22568 tablet_replica.cc:330] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: stopping tablet replica
I20260807 04:27:21.568095 22568 raft_consensus.cc:2229] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:27:21.568214 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22109
I20260807 04:27:21.568989 22568 raft_consensus.cc:2258] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:21.569655 22566 tablet_replica.cc:330] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: stopping tablet replica
I20260807 04:27:21.570319 22566 raft_consensus.cc:2229] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:21.570788 22566 raft_consensus.cc:2258] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:21.572511 22568 ts_tablet_manager.cc:1905] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:21.572511 22566 ts_tablet_manager.cc:1905] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:21.572695 22567 ts_tablet_manager.cc:1905] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:21.579408 22566 ts_tablet_manager.cc:1918] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:21.579408 22568 ts_tablet_manager.cc:1918] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:21.579579 22566 log.cc:1196] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-0/wal/wals/bba68c6d8e45461093185af96c7cfd71
I20260807 04:27:21.579703 22568 log.cc:1196] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/bba68c6d8e45461093185af96c7cfd71
I20260807 04:27:21.580159 22566 ts_tablet_manager.cc:1939] T bba68c6d8e45461093185af96c7cfd71 P 26df49ec36c745e2ab34f2bcaa0ca6fc: Deleting consensus metadata
I20260807 04:27:21.580389 22568 ts_tablet_manager.cc:1939] T bba68c6d8e45461093185af96c7cfd71 P db0e8e103a1541e4845536afab19e1b8: Deleting consensus metadata
I20260807 04:27:21.581105 22567 ts_tablet_manager.cc:1918] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:21.581279 22567 log.cc:1196] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/bba68c6d8e45461093185af96c7cfd71
I20260807 04:27:21.581938 22031 catalog_manager.cc:4841] TS 26df49ec36c745e2ab34f2bcaa0ca6fc (127.1.244.193:34717): tablet bba68c6d8e45461093185af96c7cfd71 (table test-table [id=81e32b3eb5e541f6ba42c0d88d12e7f3]) successfully deleted
I20260807 04:27:21.582003 22032 catalog_manager.cc:4841] TS db0e8e103a1541e4845536afab19e1b8 (127.1.244.195:34849): tablet bba68c6d8e45461093185af96c7cfd71 (table test-table [id=81e32b3eb5e541f6ba42c0d88d12e7f3]) successfully deleted
W20260807 04:27:21.582499 22031 catalog_manager.cc:4808] TS 26df49ec36c745e2ab34f2bcaa0ca6fc (127.1.244.193:34717): delete failed for tablet bba68c6d8e45461093185af96c7cfd71 because the tablet was not found. No further retry: Not found: Tablet not found: bba68c6d8e45461093185af96c7cfd71
I20260807 04:27:21.583767 22567 ts_tablet_manager.cc:1939] T bba68c6d8e45461093185af96c7cfd71 P 0078ef84a22248e89bf1182639f15726: Deleting consensus metadata
I20260807 04:27:21.585469 22029 catalog_manager.cc:4841] TS 0078ef84a22248e89bf1182639f15726 (127.1.244.194:44449): tablet bba68c6d8e45461093185af96c7cfd71 (table test-table [id=81e32b3eb5e541f6ba42c0d88d12e7f3]) successfully deleted
Aug 07 04:27:21 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.193@KRBTEST.COM
I20260807 04:27:21.596712 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.569653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.1.244.193:34717)
0807 04:27:21.569823 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.569839 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.579863 (+ 10024us) server_negotiation.cc:407] Connection header received
0807 04:27:21.580347 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.580353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.580436 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.580505 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:21.583349 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.584367 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.585971 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.586377 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.586511 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.590896 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:21.590912 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:21.590933 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:21.590965 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:21.593876 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.594575 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.594583 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.594585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.594715 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.595120 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.595125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.595128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.595483 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:21.595820 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.596142 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.596365 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:27:21.597512 22236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:21.686800 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22109
I20260807 04:27:21.713470 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22247
Aug 07 04:27:21 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.194@KRBTEST.COM
I20260807 04:27:21.730427 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.714386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.1.244.194:44449)
0807 04:27:21.714610 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.714617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.714640 (+ 23us) server_negotiation.cc:407] Connection header received
0807 04:27:21.714708 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.714715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.714803 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.714923 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:21.716187 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.717320 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.718592 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.719243 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.719370 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.723756 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:21.723776 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:21.723798 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:21.723846 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:21.727450 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.728128 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.728137 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.728142 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.728304 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.728655 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.728660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.728663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.729164 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:21.729449 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.729736 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.730038 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 04:27:21.731191 22379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:21.819352 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22247
I20260807 04:27:21.839947 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22390
Aug 07 04:27:21 dist-test-slave-zkpd krb5kdc[21998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.195@KRBTEST.COM
I20260807 04:27:21.857939 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:21.840767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55876 (local address 127.1.244.195:34849)
0807 04:27:21.840982 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:21.840992 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:21.841031 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:27:21.841184 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:21.841190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:21.841274 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:21.841381 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:21.842748 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.844131 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.845391 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:21.846071 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:21.846209 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:21.850797 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:21.850817 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:21.850836 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:21.850879 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:21.855026 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.855711 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.855721 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.855725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.855941 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:21.856313 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:21.856319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:21.856322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:21.856749 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:21.857054 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:21.857234 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 04:27:21.857483 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:27:21.858830 22518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:21.977685 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22390
I20260807 04:27:21.999269 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22012
I20260807 04:27:22.000447 22076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:22.111624 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22012
2026-08-07T04:27:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4733 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T04:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:22Z Disabled control of system clock
I20260807 04:27:22.198957 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35665
--webserver_interface=127.1.244.254
--webserver_port=0
--builtin_ntp_servers=127.1.244.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:35665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:27:22.539709 22577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:22.540019 22577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:22.540192 22577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:22.547472 22577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:22.547570 22577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:22.547608 22577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:22.547660 22577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:22.547751 22577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:22.556917 22577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:35665
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:35665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:22.558185 22577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:22.559983 22577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:22.572165 22582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:22.572140 22583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:22.572086 22585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:22.572945 22577 server_base.cc:1034] running on GCE node
I20260807 04:27:22.573730 22577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:22.575114 22577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:22.576344 22577 hybrid_clock.cc:648] HybridClock initialized: now 1786076842576310 us; error 45 us; skew 500 ppm
I20260807 04:27:22.576710 22577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:22.578949 22577 webserver.cc:460] Webserver started at http://127.1.244.254:44707/ using document root <none> and password file <none>
I20260807 04:27:22.579643 22577 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:22.579779 22577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:22.580173 22577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:22.583065 22577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "383912e5bf1d43f6a8866373906956de"
format_stamp: "Formatted at 2026-08-07 04:27:22 on dist-test-slave-zkpd"
server_key: "09956c3fb5300df0efafd6a1308bcad3"
server_key_iv: "563d341b0f6c22b6caf67784f761e316"
server_key_version: "encryptionkey@0"
I20260807 04:27:22.583961 22577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "383912e5bf1d43f6a8866373906956de"
format_stamp: "Formatted at 2026-08-07 04:27:22 on dist-test-slave-zkpd"
server_key: "09956c3fb5300df0efafd6a1308bcad3"
server_key_iv: "563d341b0f6c22b6caf67784f761e316"
server_key_version: "encryptionkey@0"
I20260807 04:27:22.589058 22577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 04:27:22.592410 22591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:22.593700 22577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:27:22.593890 22577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "383912e5bf1d43f6a8866373906956de"
format_stamp: "Formatted at 2026-08-07 04:27:22 on dist-test-slave-zkpd"
server_key: "09956c3fb5300df0efafd6a1308bcad3"
server_key_iv: "563d341b0f6c22b6caf67784f761e316"
server_key_version: "encryptionkey@0"
I20260807 04:27:22.594102 22577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:22.615357 22577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:22.616210 22577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:22.616503 22577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:22.635133 22577 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:35665
I20260807 04:27:22.635151 22645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:35665 every 8 connection(s)
I20260807 04:27:22.636569 22577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:22.640309 22646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:22.643675 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22577
I20260807 04:27:22.643975 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:22.644481 2003 external_mini_cluster.cc:1428] Setting key 23bf46159f1a27dac585fc8b1aa1e0f9
I20260807 04:27:22.651218 22646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de: Bootstrap starting.
I20260807 04:27:22.654765 22646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:22.656008 22646 log.cc:824] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:22.659168 22646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de: No bootstrap required, opened a new log
I20260807 04:27:22.659266 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:22.646444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.1.244.254:35665)
0807 04:27:22.647423 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:22.647433 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:22.647474 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:27:22.649819 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:22.649841 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:22.650266 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:22.650580 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:22.651252 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:22.652717 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:22.654603 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:22.655330 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:22.655466 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:22.656786 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:22.656801 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:22.656824 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:22.657593 (+ 769us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:22.657684 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:22.657705 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:22.657790 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:22.658033 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 04:27:22.658344 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":368,"threads_started":1}
I20260807 04:27:22.666508 22646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383912e5bf1d43f6a8866373906956de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35665 } }
I20260807 04:27:22.666955 22646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:22.667078 22646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383912e5bf1d43f6a8866373906956de, State: Initialized, Role: FOLLOWER
I20260807 04:27:22.667665 22646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383912e5bf1d43f6a8866373906956de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35665 } }
I20260807 04:27:22.667870 22646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:22.668036 22646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:22.668159 22646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:22.670208 22646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383912e5bf1d43f6a8866373906956de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35665 } }
I20260807 04:27:22.670629 22646 leader_election.cc:302] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 383912e5bf1d43f6a8866373906956de; no voters:
I20260807 04:27:22.671140 22646 leader_election.cc:288] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:22.671301 22651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:22.672093 22651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [term 1 LEADER]: Becoming Leader. State: Replica: 383912e5bf1d43f6a8866373906956de, State: Running, Role: LEADER
I20260807 04:27:22.672601 22651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383912e5bf1d43f6a8866373906956de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35665 } }
I20260807 04:27:22.673259 22646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:22.675771 22653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 383912e5bf1d43f6a8866373906956de. Latest consensus state: current_term: 1 leader_uuid: "383912e5bf1d43f6a8866373906956de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383912e5bf1d43f6a8866373906956de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35665 } } }
I20260807 04:27:22.676245 22653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:22.676887 22652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "383912e5bf1d43f6a8866373906956de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383912e5bf1d43f6a8866373906956de" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 35665 } } }
I20260807 04:27:22.677299 22652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:22.681208 22660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:22.686012 22660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:22.694983 22660 catalog_manager.cc:1347] Generated new cluster ID: 278ca79c4ae4407da6e62e77681668d2
I20260807 04:27:22.695087 22660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:22.713912 22660 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:22.715669 22660 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:22.729846 22660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de: Generated new TSK 0
I20260807 04:27:22.730669 22660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:22.746690 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.193:0
--local_ip_for_outbound_sockets=127.1.244.193
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35665
--builtin_ntp_servers=127.1.244.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 04:27:23.086061 22671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:23.086362 22671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:23.086500 22671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:23.093819 22671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:23.093917 22671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:23.094107 22671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.193
I20260807 04:27:23.103883 22671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.244.193
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.193
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:23.105296 22671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:23.106994 22671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:23.120018 22679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:23.119987 22677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:23.120270 22676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:23.121218 22671 server_base.cc:1034] running on GCE node
I20260807 04:27:23.121809 22671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:23.122784 22671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:23.123998 22671 hybrid_clock.cc:648] HybridClock initialized: now 1786076843123943 us; error 47 us; skew 500 ppm
I20260807 04:27:23.124321 22671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:23.126559 22671 webserver.cc:460] Webserver started at http://127.1.244.193:40171/ using document root <none> and password file <none>
I20260807 04:27:23.127261 22671 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:23.127395 22671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:23.127743 22671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:23.130673 22671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data/instance:
uuid: "378201fdfe50442b8101ff98b70d6a1d"
format_stamp: "Formatted at 2026-08-07 04:27:23 on dist-test-slave-zkpd"
server_key: "5565e84d8ccf1a96c06e786229fe0665"
server_key_iv: "109f563b70108eb8753642ad4b48470d"
server_key_version: "encryptionkey@0"
I20260807 04:27:23.131496 22671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance:
uuid: "378201fdfe50442b8101ff98b70d6a1d"
format_stamp: "Formatted at 2026-08-07 04:27:23 on dist-test-slave-zkpd"
server_key: "5565e84d8ccf1a96c06e786229fe0665"
server_key_iv: "109f563b70108eb8753642ad4b48470d"
server_key_version: "encryptionkey@0"
I20260807 04:27:23.136691 22671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 04:27:23.140197 22685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:23.141466 22671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:23.141650 22671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal
uuid: "378201fdfe50442b8101ff98b70d6a1d"
format_stamp: "Formatted at 2026-08-07 04:27:23 on dist-test-slave-zkpd"
server_key: "5565e84d8ccf1a96c06e786229fe0665"
server_key_iv: "109f563b70108eb8753642ad4b48470d"
server_key_version: "encryptionkey@0"
I20260807 04:27:23.141887 22671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:23.158444 22671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:23.159226 22671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:23.159529 22671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:23.160589 22671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:23.161999 22671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:23.162142 22671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:23.162266 22671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:23.162354 22671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:23.196532 22671 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.193:41301
I20260807 04:27:23.196590 22803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.193:41301 every 8 connection(s)
I20260807 04:27:23.198060 22671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/data/info.pb
I20260807 04:27:23.198349 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22671
I20260807 04:27:23.198520 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-0/wal/instance
I20260807 04:27:23.199224 2003 external_mini_cluster.cc:1428] Setting key 7f4fc267a6e530bcea44524803d42c4f
I20260807 04:27:23.202771 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.194:0
--local_ip_for_outbound_sockets=127.1.244.194
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35665
--builtin_ntp_servers=127.1.244.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:27:23.213811 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:23.200973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.193:57305 (local address 127.1.244.254:35665)
0807 04:27:23.201454 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:23.201461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:23.202912 (+ 1451us) server_negotiation.cc:407] Connection header received
0807 04:27:23.205753 (+ 2841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:23.205757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:23.205825 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:23.205918 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:23.207315 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.208169 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.210582 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.211006 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.211165 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:23.212824 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:23.212833 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:23.212840 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:23.212863 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:23.212916 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:23.212933 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:23.213015 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:23.213280 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 04:27:23.213386 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":259,"threads_started":1}
I20260807 04:27:23.215036 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:23.201596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35665 (local address 127.1.244.193:57305)
0807 04:27:23.202716 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:23.202768 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:23.205454 (+ 2686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:23.206193 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:23.206207 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:23.206333 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:23.207128 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:23.207157 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.208319 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.208333 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:23.210429 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:23.210447 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.211268 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.211278 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:23.211631 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:23.212471 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:23.212477 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:23.212529 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:23.212626 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:23.213039 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:23.213054 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:23.213066 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 04:27:23.213251 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 04:27:23.213672 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":304,"threads_started":1}
I20260807 04:27:23.216872 22804 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35665
I20260807 04:27:23.217223 22804 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:23.217979 22804 heartbeater.cc:511] Master 127.1.244.254:35665 requested a full tablet report, sending...
I20260807 04:27:23.220332 22610 ts_manager.cc:194] Registered new tserver with Master: 378201fdfe50442b8101ff98b70d6a1d (127.1.244.193:41301)
I20260807 04:27:23.222559 22610 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.193:57305
W20260807 04:27:23.550251 22808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:23.550529 22808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:23.550617 22808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:23.557801 22808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:23.557901 22808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:23.558020 22808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.194
I20260807 04:27:23.566823 22808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.244.194
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.194
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:23.568183 22808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:23.569902 22808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:23.583092 22816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:23.583180 22814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:23.583343 22813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:23.584209 22808 server_base.cc:1034] running on GCE node
I20260807 04:27:23.584744 22808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:23.585592 22808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:23.586799 22808 hybrid_clock.cc:648] HybridClock initialized: now 1786076843586761 us; error 29 us; skew 500 ppm
I20260807 04:27:23.587073 22808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:23.589450 22808 webserver.cc:460] Webserver started at http://127.1.244.194:33759/ using document root <none> and password file <none>
I20260807 04:27:23.590073 22808 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:23.590160 22808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:23.590443 22808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:23.593389 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data/instance:
uuid: "52b3695585c54d6c83e06b242d9564e2"
format_stamp: "Formatted at 2026-08-07 04:27:23 on dist-test-slave-zkpd"
server_key: "98f9f0ef4fcb1a9a275a5f9b805a2b4a"
server_key_iv: "f7ad6bc36f3b1f9927353bdbfe26a545"
server_key_version: "encryptionkey@0"
I20260807 04:27:23.594165 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance:
uuid: "52b3695585c54d6c83e06b242d9564e2"
format_stamp: "Formatted at 2026-08-07 04:27:23 on dist-test-slave-zkpd"
server_key: "98f9f0ef4fcb1a9a275a5f9b805a2b4a"
server_key_iv: "f7ad6bc36f3b1f9927353bdbfe26a545"
server_key_version: "encryptionkey@0"
I20260807 04:27:23.599203 22808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 04:27:23.602635 22822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:23.603874 22808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:27:23.604094 22808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal
uuid: "52b3695585c54d6c83e06b242d9564e2"
format_stamp: "Formatted at 2026-08-07 04:27:23 on dist-test-slave-zkpd"
server_key: "98f9f0ef4fcb1a9a275a5f9b805a2b4a"
server_key_iv: "f7ad6bc36f3b1f9927353bdbfe26a545"
server_key_version: "encryptionkey@0"
I20260807 04:27:23.604297 22808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:23.624183 22808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:23.625016 22808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:23.625319 22808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:23.626302 22808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:23.627779 22808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:23.627892 22808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:23.628042 22808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:23.628135 22808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:23.662532 22808 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.194:32769
I20260807 04:27:23.662626 22940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.194:32769 every 8 connection(s)
I20260807 04:27:23.663803 22808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/data/info.pb
I20260807 04:27:23.670027 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22808
I20260807 04:27:23.670194 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal/instance
I20260807 04:27:23.670624 2003 external_mini_cluster.cc:1428] Setting key b2d3dac565e130b00d7075b1aa700160
I20260807 04:27:23.673501 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.244.195:0
--local_ip_for_outbound_sockets=127.1.244.195
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35665
--builtin_ntp_servers=127.1.244.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 04:27:23.676667 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:23.666395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.194:45071 (local address 127.1.244.254:35665)
0807 04:27:23.666567 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:23.666571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:23.667803 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 04:27:23.669676 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:23.669683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:23.669775 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:23.669889 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:23.671083 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.672048 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.673624 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.674042 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.674197 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:23.675640 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:23.675648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:23.675655 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:23.675680 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:23.675726 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:23.675743 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:23.675865 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:23.676164 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 04:27:23.676273 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:23.677772 22943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:23.666815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35665 (local address 127.1.244.194:45071)
0807 04:27:23.667649 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:23.667690 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:23.669389 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:23.670150 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:23.670161 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:23.670283 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:23.670910 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:23.670929 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.672195 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.672201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:23.673482 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:23.673495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:23.674402 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:23.674408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:23.674604 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:23.675336 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:23.675339 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:23.675377 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:23.675453 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:23.675933 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:23.675950 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:23.675956 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 04:27:23.676100 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 04:27:23.676493 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":252,"threads_started":1}
I20260807 04:27:23.679220 22941 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35665
I20260807 04:27:23.679543 22941 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:23.680302 22941 heartbeater.cc:511] Master 127.1.244.254:35665 requested a full tablet report, sending...
I20260807 04:27:23.681736 22610 ts_manager.cc:194] Registered new tserver with Master: 52b3695585c54d6c83e06b242d9564e2 (127.1.244.194:32769)
I20260807 04:27:23.682938 22610 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.194:45071
W20260807 04:27:24.008056 22944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:24.008417 22944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:24.008589 22944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:24.015743 22944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:24.015870 22944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:24.016091 22944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.244.195
I20260807 04:27:24.025245 22944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.244.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.244.195
--webserver_port=0
--tserver_master_addrs=127.1.244.254:35665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.244.195
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:24.026549 22944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:24.028430 22944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:24.041603 22949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:24.041527 22952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:24.041566 22950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:24.042342 22944 server_base.cc:1034] running on GCE node
I20260807 04:27:24.042870 22944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:24.043711 22944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:24.044951 22944 hybrid_clock.cc:648] HybridClock initialized: now 1786076844044888 us; error 49 us; skew 500 ppm
I20260807 04:27:24.045313 22944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 04:27:24.047660 22944 webserver.cc:460] Webserver started at http://127.1.244.195:37869/ using document root <none> and password file <none>
I20260807 04:27:24.048424 22944 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:24.048545 22944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:24.048883 22944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:24.051676 22944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data/instance:
uuid: "94e8008d0d3f4c32983fa3f5b3bf163c"
format_stamp: "Formatted at 2026-08-07 04:27:24 on dist-test-slave-zkpd"
server_key: "a722a1a983d31a1234d6257dd1f93f5b"
server_key_iv: "db3b33a5d89f2c79b52e1b20cde13ad4"
server_key_version: "encryptionkey@0"
I20260807 04:27:24.052528 22944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance:
uuid: "94e8008d0d3f4c32983fa3f5b3bf163c"
format_stamp: "Formatted at 2026-08-07 04:27:24 on dist-test-slave-zkpd"
server_key: "a722a1a983d31a1234d6257dd1f93f5b"
server_key_iv: "db3b33a5d89f2c79b52e1b20cde13ad4"
server_key_version: "encryptionkey@0"
I20260807 04:27:24.057510 22944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:24.060998 22958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:24.062273 22944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:24.062460 22944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal
uuid: "94e8008d0d3f4c32983fa3f5b3bf163c"
format_stamp: "Formatted at 2026-08-07 04:27:24 on dist-test-slave-zkpd"
server_key: "a722a1a983d31a1234d6257dd1f93f5b"
server_key_iv: "db3b33a5d89f2c79b52e1b20cde13ad4"
server_key_version: "encryptionkey@0"
I20260807 04:27:24.062677 22944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:24.089324 22944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:24.090102 22944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:24.090348 22944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:24.091245 22944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 04:27:24.092820 22944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 04:27:24.092916 22944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:24.092993 22944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 04:27:24.093050 22944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:24.126524 22944 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.195:37293
I20260807 04:27:24.126611 23072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.195:37293 every 8 connection(s)
I20260807 04:27:24.127832 22944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/data/info.pb
I20260807 04:27:24.134379 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 22944
I20260807 04:27:24.134557 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal/instance
I20260807 04:27:24.135025 2003 external_mini_cluster.cc:1428] Setting key 8d088b83a9f930381efc0f57fbd31571
I20260807 04:27:24.140710 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.130454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:54133 (local address 127.1.244.254:35665)
0807 04:27:24.130685 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.130691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.132094 (+ 1403us) server_negotiation.cc:407] Connection header received
0807 04:27:24.133833 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.133838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.133902 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.133989 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:24.135129 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.136121 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.138213 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.138632 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.138729 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.140011 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:24.140015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:24.140018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:24.140035 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:24.140067 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:24.140079 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:24.140149 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.140406 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.140483 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 04:27:24.141695 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.131009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:35665 (local address 127.1.244.195:54133)
0807 04:27:24.131886 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.131990 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:24.133571 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:24.134205 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:24.134212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:24.134292 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:27:24.134945 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.134966 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.136308 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.136312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.138088 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.138100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.138839 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.138844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.139037 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.139662 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 04:27:24.139665 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:27:24.139717 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 04:27:24.139785 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:27:24.140202 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:27:24.140212 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:27:24.140219 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 04:27:24.140383 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 04:27:24.140642 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":262,"threads_started":1}
I20260807 04:27:24.142947 23073 heartbeater.cc:347] Connected to a master server at 127.1.244.254:35665
I20260807 04:27:24.143224 23073 heartbeater.cc:464] Registering TS with master...
I20260807 04:27:24.143996 23073 heartbeater.cc:511] Master 127.1.244.254:35665 requested a full tablet report, sending...
I20260807 04:27:24.145447 22610 ts_manager.cc:194] Registered new tserver with Master: 94e8008d0d3f4c32983fa3f5b3bf163c (127.1.244.195:37293)
I20260807 04:27:24.146458 22610 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.244.195:54133
I20260807 04:27:24.150254 2003 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 04:27:24.160136 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.154331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.1.244.254:35665)
0807 04:27:24.154524 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.154531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.155018 (+ 487us) server_negotiation.cc:407] Connection header received
0807 04:27:24.155215 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.155219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.155289 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.155371 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:24.155865 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.156737 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.158014 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.158401 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.158496 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.159433 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:24.159438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:24.159441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:24.159458 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:24.159489 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:24.159501 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:24.159569 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.159692 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.159762 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:24.163698 22610 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 04:27:24.170894 22610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 04:27:24.219280 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.184521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35578 (local address 127.1.244.193:41301)
0807 04:27:24.209874 (+ 25353us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.209887 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.209927 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:27:24.210060 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.210072 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.210410 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.211221 (+ 811us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:24.212100 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.213777 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.216546 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.218026 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.218168 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.218476 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.218740 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.218911 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25142,"thread_start_us":24953,"threads_started":1}
I20260807 04:27:24.219436 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.182962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:32769 (local address 127.0.0.1:36032)
0807 04:27:24.190775 (+ 7813us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.190839 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:24.191067 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:24.212139 (+ 21072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:24.212143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:24.212167 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:24.212716 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.212731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.214111 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.214117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.216594 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.216609 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.218389 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.218396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.218615 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.218657 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 04:27:24.218729 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 04:27:24.218876 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7457,"mutex_wait_us":56,"spinlock_wait_cycles":2304,"thread_start_us":15513,"threads_started":1}
I20260807 04:27:24.219803 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.183805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.195:37293 (local address 127.0.0.1:34016)
0807 04:27:24.211445 (+ 27640us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.211464 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:24.211622 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:24.212278 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:24.212284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:24.212316 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:24.212774 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.212788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.214982 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.214988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.217058 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.217069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.218617 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.218623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.219256 (+ 633us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.219299 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 04:27:24.219396 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 04:27:24.219513 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27517,"thread_start_us":6408,"threads_started":1}
I20260807 04:27:24.219934 23086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.183350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35578)
0807 04:27:24.198759 (+ 15409us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.198779 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:24.198911 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:24.210732 (+ 11821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:24.210743 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:24.210797 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:24.211857 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.211873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.213952 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.213957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.216342 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.216355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.218240 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.218246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.218466 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.218535 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 04:27:24.218661 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 04:27:24.218857 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15250,"mutex_wait_us":123,"thread_start_us":27711,"threads_started":1}
I20260807 04:27:24.220295 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.183149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.1.244.194:32769)
0807 04:27:24.211572 (+ 28423us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.211579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.211603 (+ 24us) server_negotiation.cc:407] Connection header received
0807 04:27:24.211678 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.211683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.211957 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.212139 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:24.212890 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.213963 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.216812 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.218164 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.219423 (+ 1259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.219655 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.219722 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.219924 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28254,"thread_start_us":28138,"threads_started":1}
I20260807 04:27:24.220779 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.185063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.1.244.195:37293)
0807 04:27:24.209292 (+ 24229us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.209305 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.211585 (+ 2280us) server_negotiation.cc:407] Connection header received
0807 04:27:24.211763 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.211770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.212049 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.212982 (+ 933us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:24.213247 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.214814 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.217235 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.218150 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.218283 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.218546 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.220191 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.220373 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23984,"thread_start_us":23813,"threads_started":1}
I20260807 04:27:24.224084 23006 tablet_service.cc:1467] Processing CreateTablet for tablet 59cad953fe474269ab39da2e0ad7b616 (DEFAULT_TABLE table=test-table [id=1a50450d8e9e4450b7e473ad05674c49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:24.224160 22871 tablet_service.cc:1467] Processing CreateTablet for tablet 59cad953fe474269ab39da2e0ad7b616 (DEFAULT_TABLE table=test-table [id=1a50450d8e9e4450b7e473ad05674c49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:24.225452 23006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59cad953fe474269ab39da2e0ad7b616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:24.225545 22871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59cad953fe474269ab39da2e0ad7b616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:24.225593 22804 heartbeater.cc:503] Master 127.1.244.254:35665 was elected leader, sending a full tablet report...
I20260807 04:27:24.225244 22735 tablet_service.cc:1467] Processing CreateTablet for tablet 59cad953fe474269ab39da2e0ad7b616 (DEFAULT_TABLE table=test-table [id=1a50450d8e9e4450b7e473ad05674c49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 04:27:24.226574 22735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59cad953fe474269ab39da2e0ad7b616. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:24.238077 23092 tablet_bootstrap.cc:492] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Bootstrap starting.
I20260807 04:27:24.240227 23093 tablet_bootstrap.cc:492] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Bootstrap starting.
I20260807 04:27:24.241866 23094 tablet_bootstrap.cc:492] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Bootstrap starting.
I20260807 04:27:24.242457 23092 tablet_bootstrap.cc:654] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:24.243724 23093 tablet_bootstrap.cc:654] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:24.243762 23092 log.cc:824] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:24.244833 23094 tablet_bootstrap.cc:654] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:24.244927 23093 log.cc:824] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:24.246064 23094 log.cc:824] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:24.247530 23092 tablet_bootstrap.cc:492] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: No bootstrap required, opened a new log
I20260807 04:27:24.247808 23092 ts_tablet_manager.cc:1397] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 04:27:24.248361 23093 tablet_bootstrap.cc:492] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: No bootstrap required, opened a new log
I20260807 04:27:24.248627 23093 ts_tablet_manager.cc:1397] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260807 04:27:24.248936 23094 tablet_bootstrap.cc:492] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: No bootstrap required, opened a new log
I20260807 04:27:24.249176 23094 ts_tablet_manager.cc:1397] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 04:27:24.257516 23094 raft_consensus.cc:348] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.257891 23094 raft_consensus.cc:374] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:24.258028 23094 raft_consensus.cc:729] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378201fdfe50442b8101ff98b70d6a1d, State: Initialized, Role: FOLLOWER
I20260807 04:27:24.258634 23094 consensus_queue.cc:260] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.258471 23092 raft_consensus.cc:348] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.258904 23092 raft_consensus.cc:374] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:24.259019 23092 raft_consensus.cc:729] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e8008d0d3f4c32983fa3f5b3bf163c, State: Initialized, Role: FOLLOWER
I20260807 04:27:24.258898 23093 raft_consensus.cc:348] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.259301 23093 raft_consensus.cc:374] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:24.259405 23093 raft_consensus.cc:729] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b3695585c54d6c83e06b242d9564e2, State: Initialized, Role: FOLLOWER
I20260807 04:27:24.259697 23092 consensus_queue.cc:260] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.260061 23093 consensus_queue.cc:260] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.260500 23094 ts_tablet_manager.cc:1428] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 04:27:24.262045 22941 heartbeater.cc:503] Master 127.1.244.254:35665 was elected leader, sending a full tablet report...
I20260807 04:27:24.262862 23093 ts_tablet_manager.cc:1428] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260807 04:27:24.263871 23073 heartbeater.cc:503] Master 127.1.244.254:35665 was elected leader, sending a full tablet report...
I20260807 04:27:24.264469 23092 ts_tablet_manager.cc:1428] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260807 04:27:24.379634 23075 tablet.cc:2369] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:24.381589 23099 raft_consensus.cc:482] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 04:27:24.381850 23099 raft_consensus.cc:504] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.383411 23099 leader_election.cc:288] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52b3695585c54d6c83e06b242d9564e2 (127.1.244.194:32769), 378201fdfe50442b8101ff98b70d6a1d (127.1.244.193:41301)
I20260807 04:27:24.389899 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.383876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.1.244.195:59279)
0807 04:27:24.384236 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.384266 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:24.384455 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:24.384933 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:24.384959 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:24.384991 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:24.385195 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.385210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.386341 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.386347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.387840 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.387852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.389139 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.389145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.389317 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.389354 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 04:27:24.389469 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 04:27:24.389610 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260807 04:27:24.389909 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.383986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:59279 (local address 127.1.244.193:41301)
0807 04:27:24.384178 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.384183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.384369 (+ 186us) server_negotiation.cc:407] Connection header received
0807 04:27:24.384629 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.384634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.384729 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.384839 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:24.385372 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.386207 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.388045 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.388987 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.389106 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.389152 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.389512 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.389634 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:27:24.390322 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.383932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.194:32769 (local address 127.1.244.195:47103)
0807 04:27:24.384715 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.384733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:27:24.384880 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:27:24.385248 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:27:24.385251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:27:24.385270 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 04:27:24.385663 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.385676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.386767 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.386771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.388302 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:27:24.388316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.389545 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.389551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:27:24.389714 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.389747 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 04:27:24.389868 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 04:27:24.390000 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":1,"thread_start_us":407,"threads_started":1}
I20260807 04:27:24.390350 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.383876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.244.195:47103 (local address 127.1.244.194:32769)
0807 04:27:24.384119 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.384126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.384842 (+ 716us) server_negotiation.cc:407] Connection header received
0807 04:27:24.385032 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.385038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.385126 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.385225 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 04:27:24.385807 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.386575 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.388483 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.389378 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.389528 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.389591 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.389932 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.390060 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 04:27:24.391204 22757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cad953fe474269ab39da2e0ad7b616" candidate_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378201fdfe50442b8101ff98b70d6a1d" is_pre_election: true
I20260807 04:27:24.391464 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cad953fe474269ab39da2e0ad7b616" candidate_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b3695585c54d6c83e06b242d9564e2" is_pre_election: true
I20260807 04:27:24.391816 22757 raft_consensus.cc:2454] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94e8008d0d3f4c32983fa3f5b3bf163c in term 0.
I20260807 04:27:24.392038 22894 raft_consensus.cc:2454] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94e8008d0d3f4c32983fa3f5b3bf163c in term 0.
I20260807 04:27:24.392617 22961 leader_election.cc:302] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378201fdfe50442b8101ff98b70d6a1d, 94e8008d0d3f4c32983fa3f5b3bf163c; no voters:
I20260807 04:27:24.393042 23099 raft_consensus.cc:2781] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 04:27:24.393193 23099 raft_consensus.cc:482] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 04:27:24.393311 23099 raft_consensus.cc:3037] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:24.395483 23099 raft_consensus.cc:504] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.396131 23099 leader_election.cc:288] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [CANDIDATE]: Term 1 election: Requested vote from peers 52b3695585c54d6c83e06b242d9564e2 (127.1.244.194:32769), 378201fdfe50442b8101ff98b70d6a1d (127.1.244.193:41301)
I20260807 04:27:24.396555 22894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cad953fe474269ab39da2e0ad7b616" candidate_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52b3695585c54d6c83e06b242d9564e2"
I20260807 04:27:24.396600 22757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cad953fe474269ab39da2e0ad7b616" candidate_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "378201fdfe50442b8101ff98b70d6a1d"
I20260807 04:27:24.396736 22894 raft_consensus.cc:3037] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:24.396822 22757 raft_consensus.cc:3037] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:24.399118 22894 raft_consensus.cc:2454] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94e8008d0d3f4c32983fa3f5b3bf163c in term 1.
I20260807 04:27:24.399354 22757 raft_consensus.cc:2454] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94e8008d0d3f4c32983fa3f5b3bf163c in term 1.
I20260807 04:27:24.399569 22960 leader_election.cc:302] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b3695585c54d6c83e06b242d9564e2, 94e8008d0d3f4c32983fa3f5b3bf163c; no voters:
I20260807 04:27:24.399868 23099 raft_consensus.cc:2781] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:24.400367 23099 raft_consensus.cc:686] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 1 LEADER]: Becoming Leader. State: Replica: 94e8008d0d3f4c32983fa3f5b3bf163c, State: Running, Role: LEADER
I20260807 04:27:24.400849 23099 consensus_queue.cc:237] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } }
I20260807 04:27:24.405520 22610 catalog_manager.cc:5495] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c reported cstate change: term changed from 0 to 1, leader changed from <none> to 94e8008d0d3f4c32983fa3f5b3bf163c (127.1.244.195). New cstate: current_term: 1 leader_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94e8008d0d3f4c32983fa3f5b3bf163c" member_type: VOTER last_known_addr { host: "127.1.244.195" port: 37293 } health_report { overall_health: HEALTHY } } }
W20260807 04:27:24.416106 22942 tablet.cc:2369] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:24.449740 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.444160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.1.244.195:37293)
0807 04:27:24.444344 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.444348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.444510 (+ 162us) server_negotiation.cc:407] Connection header received
0807 04:27:24.444677 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.444680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.444803 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.444996 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:27:24.445371 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.446223 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.447561 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.448048 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.448182 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.448666 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:27:24.449157 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:27:24.449327 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.449387 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.449475 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 04:27:24.450719 22805 tablet.cc:2369] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:27:24.458217 22894 raft_consensus.cc:1261] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 1 FOLLOWER]: Refusing update from remote peer 94e8008d0d3f4c32983fa3f5b3bf163c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:24.458222 22757 raft_consensus.cc:1261] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 1 FOLLOWER]: Refusing update from remote peer 94e8008d0d3f4c32983fa3f5b3bf163c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:27:24.459242 23102 consensus_queue.cc:1035] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [LEADER]: Connected to new peer: Peer: permanent_uuid: "52b3695585c54d6c83e06b242d9564e2" member_type: VOTER last_known_addr { host: "127.1.244.194" port: 32769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:24.459575 23099 consensus_queue.cc:1035] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [LEADER]: Connected to new peer: Peer: permanent_uuid: "378201fdfe50442b8101ff98b70d6a1d" member_type: VOTER last_known_addr { host: "127.1.244.193" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:27:24.470613 23105 mvcc.cc:205] Tried to move back new op lower bound from 7315770754890821632 to 7315770754669621248. Current Snapshot: MvccSnapshot[applied={T|T < 7315770754890821632}]
I20260807 04:27:24.485363 22608 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:24.485723 22608 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 04:27:24.489511 22608 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 383912e5bf1d43f6a8866373906956de: Sending DeleteTablet for 3 replicas of tablet 59cad953fe474269ab39da2e0ad7b616
I20260807 04:27:24.490720 23006 tablet_service.cc:1514] Processing DeleteTablet for tablet 59cad953fe474269ab39da2e0ad7b616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:24 UTC) from {username='slave'} at 127.0.0.1:34016
I20260807 04:27:24.490706 22871 tablet_service.cc:1514] Processing DeleteTablet for tablet 59cad953fe474269ab39da2e0ad7b616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:24 UTC) from {username='slave'} at 127.0.0.1:36032
I20260807 04:27:24.490773 22735 tablet_service.cc:1514] Processing DeleteTablet for tablet 59cad953fe474269ab39da2e0ad7b616 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 04:27:24 UTC) from {username='slave'} at 127.0.0.1:35578
I20260807 04:27:24.493466 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22671
I20260807 04:27:24.525156 23117 tablet_replica.cc:330] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: stopping tablet replica
I20260807 04:27:24.525781 23117 raft_consensus.cc:2229] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:24.526440 23117 raft_consensus.cc:2258] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:24.532054 23117 ts_tablet_manager.cc:1905] T 59cad953fe474269ab39da2e0ad7b616 P 378201fdfe50442b8101ff98b70d6a1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:24.538951 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.495697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.1.244.193:41301)
0807 04:27:24.495949 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.495958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.518673 (+ 22715us) server_negotiation.cc:407] Connection header received
0807 04:27:24.519286 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.519295 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.519438 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.519536 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:24.520519 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.522065 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.527012 (+ 4947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.527950 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.528081 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.536762 (+ 8681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:24.536778 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:24.536802 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:24.537731 (+ 929us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:24.537815 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:24.537848 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:24.537999 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.538296 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.538448 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:27:24.540311 22799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:24.548087 23115 tablet_replica.cc:330] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: stopping tablet replica
I20260807 04:27:24.548322 23116 tablet_replica.cc:330] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: stopping tablet replica
I20260807 04:27:24.548697 23115 raft_consensus.cc:2229] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 1 LEADER]: Raft consensus shutting down.
I20260807 04:27:24.548759 23116 raft_consensus.cc:2229] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 04:27:24.549297 23116 raft_consensus.cc:2258] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:24.549567 23115 raft_consensus.cc:2258] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 04:27:24.552805 23115 ts_tablet_manager.cc:1905] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:24.553599 23116 ts_tablet_manager.cc:1905] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 04:27:24.561698 23116 ts_tablet_manager.cc:1918] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:24.561879 23116 log.cc:1196] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-1/wal/wals/59cad953fe474269ab39da2e0ad7b616
I20260807 04:27:24.562431 23116 ts_tablet_manager.cc:1939] T 59cad953fe474269ab39da2e0ad7b616 P 52b3695585c54d6c83e06b242d9564e2: Deleting consensus metadata
I20260807 04:27:24.562606 23115 ts_tablet_manager.cc:1918] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 04:27:24.562776 23115 log.cc:1196] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Deleting WAL directory at /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786076526463677-2003-0/minicluster-data/ts-2/wal/wals/59cad953fe474269ab39da2e0ad7b616
I20260807 04:27:24.563308 23115 ts_tablet_manager.cc:1939] T 59cad953fe474269ab39da2e0ad7b616 P 94e8008d0d3f4c32983fa3f5b3bf163c: Deleting consensus metadata
I20260807 04:27:24.564374 22594 catalog_manager.cc:4841] TS 52b3695585c54d6c83e06b242d9564e2 (127.1.244.194:32769): tablet 59cad953fe474269ab39da2e0ad7b616 (table test-table [id=1a50450d8e9e4450b7e473ad05674c49]) successfully deleted
I20260807 04:27:24.565740 22593 catalog_manager.cc:4841] TS 94e8008d0d3f4c32983fa3f5b3bf163c (127.1.244.195:37293): tablet 59cad953fe474269ab39da2e0ad7b616 (table test-table [id=1a50450d8e9e4450b7e473ad05674c49]) successfully deleted
I20260807 04:27:24.660200 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22671
I20260807 04:27:24.686000 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22808
W20260807 04:27:24.686013 22595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.244.193:41301 (error 108)
I20260807 04:27:24.694916 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.694395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35596)
0807 04:27:24.694578 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.694726 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:27:24.695446 22595 catalog_manager.cc:4584] TS 378201fdfe50442b8101ff98b70d6a1d (127.1.244.193:41301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59cad953fe474269ab39da2e0ad7b616: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
I20260807 04:27:24.696110 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.687401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36034 (local address 127.1.244.194:32769)
0807 04:27:24.687585 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.687590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.687607 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:27:24.687682 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.687687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.687774 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.687876 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:24.688518 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.689668 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.691498 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.692304 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.692423 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.693905 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:24.693916 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:24.693934 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:24.694765 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:24.694823 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:24.694865 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:24.694980 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.695604 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.695721 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:27:24.696789 22936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:24.749437 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.748859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35606)
0807 04:27:24.749120 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.749237 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 04:27:24.750659 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.750244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35622)
0807 04:27:24.750417 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.750518 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 04:27:24.786916 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22808
I20260807 04:27:24.807098 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22944
I20260807 04:27:24.811190 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.810685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35638)
0807 04:27:24.810897 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.811007 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 04:27:24.812373 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.811900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35650)
0807 04:27:24.812117 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.812213 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 04:27:24.815177 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.807983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.1.244.195:37293)
0807 04:27:24.808160 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:24.808165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:24.808277 (+ 112us) server_negotiation.cc:407] Connection header received
0807 04:27:24.808489 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:24.808493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:24.808578 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:24.808673 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:24.809207 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.809965 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.811929 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:24.812420 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:24.812573 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:24.813802 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:24.813812 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:24.813826 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:24.814429 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:24.814477 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:24.814497 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:24.814623 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:24.814743 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 04:27:24.814841 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 04:27:24.816025 23068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:24.892947 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.892458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35660)
0807 04:27:24.892683 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.892768 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 04:27:24.894167 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:24.893738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35676)
0807 04:27:24.893962 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:24.894063 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 04:27:24.937222 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22944
I20260807 04:27:24.959656 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 22577
I20260807 04:27:24.960853 22641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:25.071743 23088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:25.071112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.193:41301 (local address 127.0.0.1:35678)
0807 04:27:25.071333 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 04:27:25.071469 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.193:41301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 04:27:25.072044 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 22577
2026-08-07T04:27:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2964 ms)
[----------] 6 tests from EncryptionPolicyTest (22184 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:25 dist-test-slave-zkpd krb5kdc[23126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:25 dist-test-slave-zkpd krb5kdc[23126](info): set up 2 sockets
Aug 07 04:27:25 dist-test-slave-zkpd krb5kdc[23126](info): commencing operation
krb5kdc: starting...
W20260807 04:27:26.215205 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:26 dist-test-slave-zkpd krb5kdc[23126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:26.519495 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:26.859445 23140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:26.859694 23140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:26.859791 23140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:26.866559 23140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:26.866724 23140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:26.866788 23140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:26.866850 23140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:26.866894 23140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:26.875968 23140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:26.877141 23140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:26.878330 23140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:26.889977 23149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:26.889899 23146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:26.889919 23147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:26.890904 23140 server_base.cc:1034] running on GCE node
I20260807 04:27:26.891628 23140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:26.893074 23140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:26.894308 23140 hybrid_clock.cc:648] HybridClock initialized: now 1786076846894252 us; error 42 us; skew 500 ppm
I20260807 04:27:26.894712 23140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:26 dist-test-slave-zkpd krb5kdc[23126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:26.902236 23140 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:26.903760 23140 webserver.cc:460] Webserver started at http://127.0.0.1:38635/ using document root <none> and password file <none>
I20260807 04:27:26.904500 23140 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:26.904646 23140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:26.904979 23140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:26.908097 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "076cc9b1aab542ceb55bd69a7d585c11"
format_stamp: "Formatted at 2026-08-07 04:27:26 on dist-test-slave-zkpd"
server_key: "da898310a29da016dd42ab0a51971fa5"
server_key_iv: "1531746fd5e3f58593a5ed2b0ae6ec85"
server_key_version: "encryptionkey@0"
I20260807 04:27:26.908877 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "076cc9b1aab542ceb55bd69a7d585c11"
format_stamp: "Formatted at 2026-08-07 04:27:26 on dist-test-slave-zkpd"
server_key: "da898310a29da016dd42ab0a51971fa5"
server_key_iv: "1531746fd5e3f58593a5ed2b0ae6ec85"
server_key_version: "encryptionkey@0"
I20260807 04:27:26.913913 23140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:26.917477 23157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:26.918771 23140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 04:27:26.918936 23140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "076cc9b1aab542ceb55bd69a7d585c11"
format_stamp: "Formatted at 2026-08-07 04:27:26 on dist-test-slave-zkpd"
server_key: "da898310a29da016dd42ab0a51971fa5"
server_key_iv: "1531746fd5e3f58593a5ed2b0ae6ec85"
server_key_version: "encryptionkey@0"
I20260807 04:27:26.919176 23140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:26.942096 23140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:26.946501 23140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:26.946812 23140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:26.965229 23140 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45773
I20260807 04:27:26.965267 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45773 every 8 connection(s)
I20260807 04:27:26.966570 23140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:26.968660 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23140
I20260807 04:27:26.968884 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:26.969542 2003 external_mini_cluster.cc:1428] Setting key f0a3a93a88b78a3cf76881207bbd358f
I20260807 04:27:26.970592 23211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:27:26 dist-test-slave-zkpd krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:27:26.985080 23211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11: Bootstrap starting.
I20260807 04:27:26.989270 23211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:26.990479 23211 log.cc:824] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:26.993865 23211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11: No bootstrap required, opened a new log
I20260807 04:27:26.993943 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:26.971432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37973 (local address 127.0.0.1:45773)
0807 04:27:26.972225 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:26.972236 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:26.972275 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:27:26.973874 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:26.973893 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:26.974274 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:26.974602 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:26.976587 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:26.977922 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:26.979713 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:26.980415 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:26.980541 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:26.985330 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:26.985340 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:26.985360 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:26.985404 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:26.989268 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:26.990035 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:26.990044 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:26.990055 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:26.990296 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:26.990740 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:26.990745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:26.990748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:26.991413 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:26.991653 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:26.992301 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 04:27:26.992615 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":360,"threads_started":1}
I20260807 04:27:26.998032 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:27.002293 23211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076cc9b1aab542ceb55bd69a7d585c11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45773 } }
I20260807 04:27:27.002753 23211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:27.002893 23211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 076cc9b1aab542ceb55bd69a7d585c11, State: Initialized, Role: FOLLOWER
I20260807 04:27:27.003545 23211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076cc9b1aab542ceb55bd69a7d585c11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45773 } }
I20260807 04:27:27.003854 23211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:27.004040 23211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:27.004222 23211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:27.006938 23211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076cc9b1aab542ceb55bd69a7d585c11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45773 } }
I20260807 04:27:27.007403 23211 leader_election.cc:302] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 076cc9b1aab542ceb55bd69a7d585c11; no voters:
I20260807 04:27:27.008100 23211 leader_election.cc:288] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:27.008217 23221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:27.010411 23211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:27.012068 23221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [term 1 LEADER]: Becoming Leader. State: Replica: 076cc9b1aab542ceb55bd69a7d585c11, State: Running, Role: LEADER
I20260807 04:27:27.012804 23221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076cc9b1aab542ceb55bd69a7d585c11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45773 } }
I20260807 04:27:27.016358 23222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "076cc9b1aab542ceb55bd69a7d585c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076cc9b1aab542ceb55bd69a7d585c11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45773 } } }
I20260807 04:27:27.016795 23222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:27.016331 23223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 076cc9b1aab542ceb55bd69a7d585c11. Latest consensus state: current_term: 1 leader_uuid: "076cc9b1aab542ceb55bd69a7d585c11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "076cc9b1aab542ceb55bd69a7d585c11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45773 } } }
I20260807 04:27:27.017408 23223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:27.021752 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:27.002742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33821 (local address 127.0.0.1:45773)
0807 04:27:27.002935 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:27.002941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:27.003478 (+ 537us) server_negotiation.cc:407] Connection header received
0807 04:27:27.003592 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:27.003597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:27.003669 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:27.003783 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:27.005284 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:27.006462 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:27.007784 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:27.008797 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:27.009514 (+ 717us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:27.012023 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:27.012029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:27.012038 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:27.012086 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:27.016438 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:27.017695 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:27.017702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:27.017708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:27.017940 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:27.018678 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:27.018687 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:27.018692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:27.019200 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:27.020742 (+ 1542us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:27.020984 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 04:27:27.021230 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 04:27:27.024747 23232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:27.030973 23232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:27.040076 23232 catalog_manager.cc:1347] Generated new cluster ID: 55d7ef6758dd4bf9bfd12bd9b38b4b52
I20260807 04:27:27.040176 23232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:27.056918 23232 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:27.058156 23232 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:27.065522 23232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 076cc9b1aab542ceb55bd69a7d585c11: Generated new TSK 0
I20260807 04:27:27.066290 23232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:27.079394 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23140
I20260807 04:27:27.080960 23206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:27.172758 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23140
W20260807 04:27:27.192839 23124 connection.cc:535] client connection to 127.0.0.1:45773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45773): Connection reset by peer (error 104)
2026-08-07T04:27:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2090 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:27 dist-test-slave-zkpd krb5kdc[23246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:27 dist-test-slave-zkpd krb5kdc[23246](info): set up 2 sockets
Aug 07 04:27:27 dist-test-slave-zkpd krb5kdc[23246](info): commencing operation
krb5kdc: starting...
W20260807 04:27:28.323437 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:28 dist-test-slave-zkpd krb5kdc[23246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:28.637558 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:28.983556 23260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:28.983901 23260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:28.984104 23260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:28.991211 23260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:28.991367 23260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:28.991489 23260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:28.991598 23260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:28.991698 23260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:29.001711 23260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:29.003069 23260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:29.004432 23260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:29.016645 23269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:29.016620 23267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:29.017176 23266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:29.017606 23260 server_base.cc:1034] running on GCE node
I20260807 04:27:29.018323 23260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:29.019624 23260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:29.020844 23260 hybrid_clock.cc:648] HybridClock initialized: now 1786076849020783 us; error 49 us; skew 500 ppm
I20260807 04:27:29.021220 23260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:29 dist-test-slave-zkpd krb5kdc[23246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:29.024827 23260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:29.026304 23260 webserver.cc:460] Webserver started at http://127.0.0.1:42641/ using document root <none> and password file <none>
I20260807 04:27:29.027083 23260 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:29.027217 23260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:29.027547 23260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:29.030622 23260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "a7b37382fe01421cb407fb9d4415c3cd"
format_stamp: "Formatted at 2026-08-07 04:27:29 on dist-test-slave-zkpd"
server_key: "9078655cf30f9b1527cbb78c7396fe91"
server_key_iv: "da25a911fced275888f7f3de039412cb"
server_key_version: "encryptionkey@0"
I20260807 04:27:29.031412 23260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "a7b37382fe01421cb407fb9d4415c3cd"
format_stamp: "Formatted at 2026-08-07 04:27:29 on dist-test-slave-zkpd"
server_key: "9078655cf30f9b1527cbb78c7396fe91"
server_key_iv: "da25a911fced275888f7f3de039412cb"
server_key_version: "encryptionkey@0"
I20260807 04:27:29.036576 23260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:27:29.040110 23277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:29.041354 23260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:29.041529 23260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "a7b37382fe01421cb407fb9d4415c3cd"
format_stamp: "Formatted at 2026-08-07 04:27:29 on dist-test-slave-zkpd"
server_key: "9078655cf30f9b1527cbb78c7396fe91"
server_key_iv: "da25a911fced275888f7f3de039412cb"
server_key_version: "encryptionkey@0"
I20260807 04:27:29.041728 23260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:29.078231 23260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:29.082202 23260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:29.082507 23260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:29.101321 23260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42193
I20260807 04:27:29.101362 23330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42193 every 8 connection(s)
I20260807 04:27:29.102715 23260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:29.106539 23331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:29.108258 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23260
I20260807 04:27:29.108445 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:29.108940 2003 external_mini_cluster.cc:1428] Setting key ba524f76d925b13f0de19da659bcd4bb
I20260807 04:27:29.121628 23331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd: Bootstrap starting.
I20260807 04:27:29.123850 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:29.110804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37127 (local address 127.0.0.1:42193)
0807 04:27:29.111622 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:29.111637 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:29.111699 (+ 62us) server_negotiation.cc:407] Connection header received
0807 04:27:29.113487 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:29.113517 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:29.114070 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:29.114552 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:29.115094 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:29.116995 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:29.118396 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:29.119515 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:29.119687 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:29.121000 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:29.121014 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:29.121033 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:29.121748 (+ 715us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:29.121847 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:29.121883 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:29.122052 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:29.122341 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 04:27:29.122534 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":313,"threads_started":1}
I20260807 04:27:29.125689 23331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:29.126984 23331 log.cc:824] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:29.128111 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:29.130461 23331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd: No bootstrap required, opened a new log
I20260807 04:27:29.140307 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:29.133445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50643 (local address 127.0.0.1:42193)
0807 04:27:29.133638 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:29.133645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:29.134151 (+ 506us) server_negotiation.cc:407] Connection header received
0807 04:27:29.134305 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:29.134311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:29.134389 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:29.134510 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:29.135009 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:29.136183 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:29.137534 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:29.138235 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:29.138369 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:29.139484 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:29.139490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:29.139495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:29.139516 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:29.139577 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:29.139600 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:29.139714 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:29.139884 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 04:27:29.139986 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:27:29.140551 23331 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b37382fe01421cb407fb9d4415c3cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260807 04:27:29.140990 23331 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:29.141115 23331 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7b37382fe01421cb407fb9d4415c3cd, State: Initialized, Role: FOLLOWER
I20260807 04:27:29.141741 23331 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b37382fe01421cb407fb9d4415c3cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260807 04:27:29.141997 23331 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:29.142167 23331 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:29.142343 23331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:29.144692 23331 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b37382fe01421cb407fb9d4415c3cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260807 04:27:29.145270 23331 leader_election.cc:302] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7b37382fe01421cb407fb9d4415c3cd; no voters:
I20260807 04:27:29.145977 23331 leader_election.cc:288] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:29.146137 23341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:29.146929 23341 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [term 1 LEADER]: Becoming Leader. State: Replica: a7b37382fe01421cb407fb9d4415c3cd, State: Running, Role: LEADER
I20260807 04:27:29.147557 23341 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b37382fe01421cb407fb9d4415c3cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } }
I20260807 04:27:29.148357 23331 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:29.150768 23343 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7b37382fe01421cb407fb9d4415c3cd. Latest consensus state: current_term: 1 leader_uuid: "a7b37382fe01421cb407fb9d4415c3cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b37382fe01421cb407fb9d4415c3cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } } }
I20260807 04:27:29.150796 23342 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7b37382fe01421cb407fb9d4415c3cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b37382fe01421cb407fb9d4415c3cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42193 } } }
I20260807 04:27:29.151248 23343 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:29.151260 23342 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:29.153296 23346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:29.159166 23346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:29.168052 23346 catalog_manager.cc:1347] Generated new cluster ID: 95af36c02b69402794a5616a35a416cd
I20260807 04:27:29.168164 23346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:29.175608 23346 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:29.177094 23346 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:29.182569 23346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7b37382fe01421cb407fb9d4415c3cd: Generated new TSK 0
I20260807 04:27:29.183344 23346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:29.189985 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23260
W20260807 04:27:29.191376 23326 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37127
W20260807 04:27:29.192147 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23260: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:29.192351 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23260
W20260807 04:27:29.212647 23244 connection.cc:535] client connection to 127.0.0.1:42193 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42193): Connection reset by peer (error 104)
2026-08-07T04:27:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:29 dist-test-slave-zkpd krb5kdc[23362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:29 dist-test-slave-zkpd krb5kdc[23362](info): set up 2 sockets
Aug 07 04:27:29 dist-test-slave-zkpd krb5kdc[23362](info): commencing operation
krb5kdc: starting...
W20260807 04:27:30.323056 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:30 dist-test-slave-zkpd krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:30.602742 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:30.932222 23375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:30.932555 23375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:30.932714 23375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:30.939740 23375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:30.939927 23375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:30.940037 23375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:30.940145 23375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:30.940246 23375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:30.949548 23375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:30.950776 23375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:30.952104 23375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:30.963965 23381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:30.963915 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:30.964002 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:30.965003 23375 server_base.cc:1034] running on GCE node
I20260807 04:27:30.965775 23375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:30.967074 23375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:30.968302 23375 hybrid_clock.cc:648] HybridClock initialized: now 1786076850968259 us; error 55 us; skew 500 ppm
I20260807 04:27:30.968639 23375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:30 dist-test-slave-zkpd krb5kdc[23362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076850, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:30.971652 23375 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:30.973040 23375 webserver.cc:460] Webserver started at http://127.0.0.1:36181/ using document root <none> and password file <none>
I20260807 04:27:30.973641 23375 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:30.973753 23375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:30.974017 23375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:30.976919 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "1b26144782db49dfb3b48f45778b0378"
format_stamp: "Formatted at 2026-08-07 04:27:30 on dist-test-slave-zkpd"
server_key: "bbbb8543c7c78ae008c2af2e8f835bab"
server_key_iv: "1c5d1373ec37b0ddea1369ff3e475d2e"
server_key_version: "encryptionkey@0"
I20260807 04:27:30.977620 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1b26144782db49dfb3b48f45778b0378"
format_stamp: "Formatted at 2026-08-07 04:27:30 on dist-test-slave-zkpd"
server_key: "bbbb8543c7c78ae008c2af2e8f835bab"
server_key_iv: "1c5d1373ec37b0ddea1369ff3e475d2e"
server_key_version: "encryptionkey@0"
I20260807 04:27:30.982415 23375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 04:27:30.985653 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:30.986862 23375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 04:27:30.987042 23375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "1b26144782db49dfb3b48f45778b0378"
format_stamp: "Formatted at 2026-08-07 04:27:30 on dist-test-slave-zkpd"
server_key: "bbbb8543c7c78ae008c2af2e8f835bab"
server_key_iv: "1c5d1373ec37b0ddea1369ff3e475d2e"
server_key_version: "encryptionkey@0"
I20260807 04:27:30.987231 23375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:30.992703 23375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:30.996652 23375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:30.996976 23375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:31.015681 23375 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44829
I20260807 04:27:31.015756 23442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44829 every 8 connection(s)
I20260807 04:27:31.017076 23375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:31.018811 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23375
I20260807 04:27:31.019013 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:31.019578 2003 external_mini_cluster.cc:1428] Setting key 9191af69ededa0ca22e88504a5a97181
I20260807 04:27:31.021474 23443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:31.027128 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:31.021332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33783 (local address 127.0.0.1:44829)
0807 04:27:31.022147 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:31.022157 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:31.022196 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:27:31.023740 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:31.023760 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:31.024152 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:31.024486 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:31.025517 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:31.025522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:31.025534 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:31.025549 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:31.025630 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:31.025650 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:31.025812 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:31.025941 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 04:27:31.026133 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":375,"threads_started":1}
I20260807 04:27:31.030632 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:31.034265 23443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378: Bootstrap starting.
I20260807 04:27:31.038378 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:31.036016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33199 (local address 127.0.0.1:44829)
0807 04:27:31.036218 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:31.036223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:31.036515 (+ 292us) server_negotiation.cc:407] Connection header received
0807 04:27:31.036676 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:31.036680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:31.036729 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:31.036807 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:31.037730 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:31.037734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:31.037736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:31.037750 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:31.037786 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:31.037799 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:31.037885 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:31.038032 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 04:27:31.038074 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 04:27:31.038399 23443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:31.039557 23443 log.cc:824] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:31.042296 23443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378: No bootstrap required, opened a new log
I20260807 04:27:31.049330 23443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b26144782db49dfb3b48f45778b0378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260807 04:27:31.049721 23443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:31.049841 23443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b26144782db49dfb3b48f45778b0378, State: Initialized, Role: FOLLOWER
I20260807 04:27:31.050390 23443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b26144782db49dfb3b48f45778b0378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260807 04:27:31.050637 23443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:31.050778 23443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:31.050952 23443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:31.053256 23443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b26144782db49dfb3b48f45778b0378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260807 04:27:31.053776 23443 leader_election.cc:302] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b26144782db49dfb3b48f45778b0378; no voters:
I20260807 04:27:31.054318 23443 leader_election.cc:288] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:31.054458 23453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:31.055464 23453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [term 1 LEADER]: Becoming Leader. State: Replica: 1b26144782db49dfb3b48f45778b0378, State: Running, Role: LEADER
I20260807 04:27:31.056114 23453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b26144782db49dfb3b48f45778b0378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } }
I20260807 04:27:31.056463 23443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:31.059044 23455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b26144782db49dfb3b48f45778b0378. Latest consensus state: current_term: 1 leader_uuid: "1b26144782db49dfb3b48f45778b0378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b26144782db49dfb3b48f45778b0378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } } }
I20260807 04:27:31.059592 23455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:31.059547 23454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b26144782db49dfb3b48f45778b0378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b26144782db49dfb3b48f45778b0378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44829 } } }
I20260807 04:27:31.060048 23454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:31.066313 23462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:31.079305 23462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:31.088317 23462 catalog_manager.cc:1347] Generated new cluster ID: f1f6f9e4710a4304bf833de27e52a8bb
I20260807 04:27:31.088438 23462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:31.131819 23462 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:31.132275 23462 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:31.141240 23462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b26144782db49dfb3b48f45778b0378: Generated new TSK 0
I20260807 04:27:31.142019 23462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:31.155750 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23375
W20260807 04:27:31.156836 23438 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33783
W20260807 04:27:31.157698 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23375: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:31.157846 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23375
2026-08-07T04:27:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1964 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:31 dist-test-slave-zkpd krb5kdc[23475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:31 dist-test-slave-zkpd krb5kdc[23475](info): set up 2 sockets
Aug 07 04:27:31 dist-test-slave-zkpd krb5kdc[23475](info): commencing operation
krb5kdc: starting...
W20260807 04:27:32.278959 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:32 dist-test-slave-zkpd krb5kdc[23475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:32.570724 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:32.905781 23488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:32.906046 23488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:32.906137 23488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:32.913122 23488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:32.913219 23488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:32.913272 23488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:32.913331 23488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:32.913374 23488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:32.922796 23488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:32.923913 23488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:32.925125 23488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:32.937124 23495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:32.937124 23497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:32.937271 23494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:32.941190 23488 server_base.cc:1034] running on GCE node
I20260807 04:27:32.941959 23488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:32.943212 23488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:32.944437 23488 hybrid_clock.cc:648] HybridClock initialized: now 1786076852944371 us; error 24 us; skew 500 ppm
I20260807 04:27:32.944774 23488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:32 dist-test-slave-zkpd krb5kdc[23475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076852, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:32.952234 23488 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:32.953665 23488 webserver.cc:460] Webserver started at http://127.0.0.1:46153/ using document root <none> and password file <none>
I20260807 04:27:32.954375 23488 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:32.954512 23488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:32.954828 23488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:32.957789 23488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "1982f5debca44c8e9bc201b346969563"
format_stamp: "Formatted at 2026-08-07 04:27:32 on dist-test-slave-zkpd"
server_key: "04739190fb14480abcee70079647bea0"
server_key_iv: "384576dbb4a5d611c9bd6a0e6a5caef1"
server_key_version: "encryptionkey@0"
I20260807 04:27:32.958545 23488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1982f5debca44c8e9bc201b346969563"
format_stamp: "Formatted at 2026-08-07 04:27:32 on dist-test-slave-zkpd"
server_key: "04739190fb14480abcee70079647bea0"
server_key_iv: "384576dbb4a5d611c9bd6a0e6a5caef1"
server_key_version: "encryptionkey@0"
I20260807 04:27:32.963557 23488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 04:27:32.966928 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:32.968289 23488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:32.968482 23488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "1982f5debca44c8e9bc201b346969563"
format_stamp: "Formatted at 2026-08-07 04:27:32 on dist-test-slave-zkpd"
server_key: "04739190fb14480abcee70079647bea0"
server_key_iv: "384576dbb4a5d611c9bd6a0e6a5caef1"
server_key_version: "encryptionkey@0"
I20260807 04:27:32.968688 23488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:32.996130 23488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:33.000313 23488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:33.000599 23488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:33.019017 23488 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33881
I20260807 04:27:33.019065 23558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33881 every 8 connection(s)
I20260807 04:27:33.020336 23488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:33.023782 23559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:33.028704 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23488
I20260807 04:27:33.028898 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:33.029314 2003 external_mini_cluster.cc:1428] Setting key 2e59bbbad13e622096c45a2dbc6d948a
I20260807 04:27:33.034631 23559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563: Bootstrap starting.
I20260807 04:27:33.038237 23559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:33.039471 23559 log.cc:824] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:33.042965 23559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563: No bootstrap required, opened a new log
Aug 07 04:27:33 dist-test-slave-zkpd krb5kdc[23475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:27:33.050669 23559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1982f5debca44c8e9bc201b346969563" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33881 } }
I20260807 04:27:33.051070 23559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:33.051208 23559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1982f5debca44c8e9bc201b346969563, State: Initialized, Role: FOLLOWER
I20260807 04:27:33.051873 23559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1982f5debca44c8e9bc201b346969563" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33881 } }
I20260807 04:27:33.052219 23559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:33.052383 23559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:33.052582 23559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:33.053478 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:33.031099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59313 (local address 127.0.0.1:33881)
0807 04:27:33.032054 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:33.032065 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:33.032107 (+ 42us) server_negotiation.cc:407] Connection header received
0807 04:27:33.033867 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:33.033889 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:33.034241 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:33.034551 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:33.036469 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:33.037753 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:33.039692 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:33.040785 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:33.040974 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:33.045784 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:33.045792 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:33.045805 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:33.045853 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:33.049122 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:33.049856 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:33.049863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:33.049868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:33.050032 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:33.050427 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:33.050433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:33.050436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:33.050974 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:33.051312 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:33.051772 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 04:27:33.052309 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":425,"threads_started":1}
I20260807 04:27:33.054987 23559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1982f5debca44c8e9bc201b346969563" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33881 } }
I20260807 04:27:33.055464 23559 leader_election.cc:302] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1982f5debca44c8e9bc201b346969563; no voters:
I20260807 04:27:33.056185 23559 leader_election.cc:288] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:33.056280 23564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:33.057256 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:33.058638 23559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:33.063268 23564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [term 1 LEADER]: Becoming Leader. State: Replica: 1982f5debca44c8e9bc201b346969563, State: Running, Role: LEADER
I20260807 04:27:33.063951 23564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1982f5debca44c8e9bc201b346969563" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33881 } }
I20260807 04:27:33.067288 23565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1982f5debca44c8e9bc201b346969563" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1982f5debca44c8e9bc201b346969563" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33881 } } }
I20260807 04:27:33.067754 23565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:33.068317 23566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1982f5debca44c8e9bc201b346969563. Latest consensus state: current_term: 1 leader_uuid: "1982f5debca44c8e9bc201b346969563" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1982f5debca44c8e9bc201b346969563" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33881 } } }
I20260807 04:27:33.068640 23566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:33.076678 23579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:33.082345 23579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:33.087685 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:33.066893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47915 (local address 127.0.0.1:33881)
0807 04:27:33.068525 (+ 1632us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:33.068534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:33.068583 (+ 49us) server_negotiation.cc:407] Connection header received
0807 04:27:33.068781 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:33.068789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:33.068900 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:33.069020 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:33.071646 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:33.073421 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:33.075801 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:33.076612 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:33.076764 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:33.080092 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:33.080105 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:33.080113 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:33.080154 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:33.083946 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:33.084755 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:33.084760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:33.084763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:33.084913 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:33.085374 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:33.085379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:33.085382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:33.085796 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:33.086035 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:33.087314 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0807 04:27:33.087464 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1468}
I20260807 04:27:33.091324 23579 catalog_manager.cc:1347] Generated new cluster ID: f0d9b61616644c5c9f56376b16eb67a7
I20260807 04:27:33.091444 23579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:33.104377 23579 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:33.105593 23579 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:33.110412 23579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1982f5debca44c8e9bc201b346969563: Generated new TSK 0
I20260807 04:27:33.111305 23579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:33.124687 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23488
I20260807 04:27:33.125635 23554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:33.213369 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23488
2026-08-07T04:27:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:33 dist-test-slave-zkpd krb5kdc[23593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:33 dist-test-slave-zkpd krb5kdc[23593](info): set up 2 sockets
Aug 07 04:27:33 dist-test-slave-zkpd krb5kdc[23593](info): commencing operation
krb5kdc: starting...
W20260807 04:27:34.338788 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:34 dist-test-slave-zkpd krb5kdc[23593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:34.623029 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:34.958974 23607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:34.959329 23607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:34.959492 23607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:34.966547 23607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:34.966698 23607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:34.966800 23607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:34.966907 23607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:34.966980 23607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:34.976539 23607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:34.977787 23607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:34.979063 23607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:34.990772 23612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:34.990728 23613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:34.990749 23615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:34.991504 23607 server_base.cc:1034] running on GCE node
I20260807 04:27:34.992264 23607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:34.993522 23607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:34.994725 23607 hybrid_clock.cc:648] HybridClock initialized: now 1786076854994689 us; error 29 us; skew 500 ppm
I20260807 04:27:34.995064 23607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:34 dist-test-slave-zkpd krb5kdc[23593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076854, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:34.998167 23607 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:34.999642 23607 webserver.cc:460] Webserver started at http://127.0.0.1:34069/ using document root <none> and password file <none>
I20260807 04:27:35.000399 23607 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:35.000553 23607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:35.000813 23607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:35.003700 23607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "27474b0589034411aab840883268d68a"
format_stamp: "Formatted at 2026-08-07 04:27:34 on dist-test-slave-zkpd"
server_key: "0d409298233a8ebec9cd7712e7498bcd"
server_key_iv: "68f8b5f9aac839aecc6102aa0eee8128"
server_key_version: "encryptionkey@0"
I20260807 04:27:35.004503 23607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "27474b0589034411aab840883268d68a"
format_stamp: "Formatted at 2026-08-07 04:27:34 on dist-test-slave-zkpd"
server_key: "0d409298233a8ebec9cd7712e7498bcd"
server_key_iv: "68f8b5f9aac839aecc6102aa0eee8128"
server_key_version: "encryptionkey@0"
I20260807 04:27:35.009382 23607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 04:27:35.012735 23623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:35.013981 23607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:35.014168 23607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "27474b0589034411aab840883268d68a"
format_stamp: "Formatted at 2026-08-07 04:27:34 on dist-test-slave-zkpd"
server_key: "0d409298233a8ebec9cd7712e7498bcd"
server_key_iv: "68f8b5f9aac839aecc6102aa0eee8128"
server_key_version: "encryptionkey@0"
I20260807 04:27:35.014364 23607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:35.028482 23607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:35.032923 23607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:35.033214 23607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:35.051800 23607 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36575
I20260807 04:27:35.051882 23675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36575 every 8 connection(s)
I20260807 04:27:35.053156 23607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:35.056797 23676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:35.061796 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23607
I20260807 04:27:35.061995 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:35.062440 2003 external_mini_cluster.cc:1428] Setting key 276ab8b20910a494e3e75d38cd63a1e7
I20260807 04:27:35.067615 23676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a: Bootstrap starting.
I20260807 04:27:35.070717 23676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:35.071794 23676 log.cc:824] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:35.074796 23676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a: No bootstrap required, opened a new log
I20260807 04:27:35.075220 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:35.064205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40777 (local address 127.0.0.1:36575)
0807 04:27:35.065051 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:35.065061 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:35.065107 (+ 46us) server_negotiation.cc:407] Connection header received
0807 04:27:35.066734 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:35.066753 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:35.067107 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:35.067470 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:35.068038 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:35.069364 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:35.070736 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:35.071469 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:35.071671 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:35.072939 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:35.072952 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:35.072975 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:35.073540 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:35.073612 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:35.073636 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:35.073734 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:35.073975 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 04:27:35.074246 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":388,"threads_started":1}
I20260807 04:27:35.078924 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:35.083248 23676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27474b0589034411aab840883268d68a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36575 } }
I20260807 04:27:35.083690 23676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:35.083809 23676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27474b0589034411aab840883268d68a, State: Initialized, Role: FOLLOWER
I20260807 04:27:35.084578 23676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27474b0589034411aab840883268d68a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36575 } }
I20260807 04:27:35.084841 23676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:35.085021 23676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:35.085216 23676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:35.087823 23676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27474b0589034411aab840883268d68a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36575 } }
I20260807 04:27:35.088426 23676 leader_election.cc:302] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27474b0589034411aab840883268d68a; no voters:
I20260807 04:27:35.089094 23676 leader_election.cc:288] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:35.089247 23686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:35.090325 23679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:35.083260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59621 (local address 127.0.0.1:36575)
0807 04:27:35.083449 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:35.083454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:35.083931 (+ 477us) server_negotiation.cc:407] Connection header received
0807 04:27:35.084137 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:35.084145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:35.084222 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:35.084344 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:35.084857 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:35.085982 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:35.087251 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:35.087986 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:35.088114 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:35.089243 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:35.089253 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:35.089257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:35.089278 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:35.089320 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:35.089344 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:35.089471 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:35.089737 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 04:27:35.089874 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 04:27:35.091071 23686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [term 1 LEADER]: Becoming Leader. State: Replica: 27474b0589034411aab840883268d68a, State: Running, Role: LEADER
I20260807 04:27:35.091650 23686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27474b0589034411aab840883268d68a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36575 } }
I20260807 04:27:35.093394 23686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27474b0589034411aab840883268d68a. Latest consensus state: current_term: 1 leader_uuid: "27474b0589034411aab840883268d68a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27474b0589034411aab840883268d68a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36575 } } }
I20260807 04:27:35.093751 23686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:35.094286 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27474b0589034411aab840883268d68a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27474b0589034411aab840883268d68a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36575 } } }
I20260807 04:27:35.094719 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:35.097734 23676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:35.100940 23690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:35.106395 23690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:35.115032 23690 catalog_manager.cc:1347] Generated new cluster ID: 60b44abf39314fb69d50e0c35c188e62
I20260807 04:27:35.115128 23690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:35.132879 23690 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:35.134678 23690 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:35.146852 23690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27474b0589034411aab840883268d68a: Generated new TSK 0
I20260807 04:27:35.147660 23690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:35.159972 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23607
W20260807 04:27:35.161052 23671 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40777
W20260807 04:27:35.161806 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23607: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:35.161957 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23607
2026-08-07T04:27:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1948 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:35 dist-test-slave-zkpd krb5kdc[23710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:35 dist-test-slave-zkpd krb5kdc[23710](info): set up 2 sockets
Aug 07 04:27:35 dist-test-slave-zkpd krb5kdc[23710](info): commencing operation
krb5kdc: starting...
W20260807 04:27:36.299880 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:36 dist-test-slave-zkpd krb5kdc[23710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:36.583530 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:36.918715 23723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:36.918974 23723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:36.919149 23723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:36.926231 23723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:36.926347 23723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:36.926393 23723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:36.926489 23723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:36.926558 23723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:36.936053 23723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:36.937307 23723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:36.938581 23723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:36.950007 23729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:36.950042 23728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:36.950078 23731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:36.951176 23723 server_base.cc:1034] running on GCE node
I20260807 04:27:36.951944 23723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:36.953299 23723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:36.954541 23723 hybrid_clock.cc:648] HybridClock initialized: now 1786076856954508 us; error 40 us; skew 500 ppm
I20260807 04:27:36.954891 23723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:36 dist-test-slave-zkpd krb5kdc[23710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076856, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:36.957929 23723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:36.959316 23723 webserver.cc:460] Webserver started at http://127.0.0.1:38935/ using document root <none> and password file <none>
I20260807 04:27:36.959990 23723 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:36.960125 23723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:36.960445 23723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:36.963423 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "03301b9ad61740cb86c5001186f4f6f9"
format_stamp: "Formatted at 2026-08-07 04:27:36 on dist-test-slave-zkpd"
server_key: "6a4dec59e017b32c79ce4905fbbf750f"
server_key_iv: "fa5209a3b2605d2b2caa134a2d0cf881"
server_key_version: "encryptionkey@0"
I20260807 04:27:36.964210 23723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "03301b9ad61740cb86c5001186f4f6f9"
format_stamp: "Formatted at 2026-08-07 04:27:36 on dist-test-slave-zkpd"
server_key: "6a4dec59e017b32c79ce4905fbbf750f"
server_key_iv: "fa5209a3b2605d2b2caa134a2d0cf881"
server_key_version: "encryptionkey@0"
I20260807 04:27:36.969134 23723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:27:36.972469 23738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:36.973713 23723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:27:36.973896 23723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "03301b9ad61740cb86c5001186f4f6f9"
format_stamp: "Formatted at 2026-08-07 04:27:36 on dist-test-slave-zkpd"
server_key: "6a4dec59e017b32c79ce4905fbbf750f"
server_key_iv: "fa5209a3b2605d2b2caa134a2d0cf881"
server_key_version: "encryptionkey@0"
I20260807 04:27:36.974094 23723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:36.979790 23723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:36.983515 23723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:36.983816 23723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:37.002413 23723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33369
I20260807 04:27:37.002491 23790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33369 every 8 connection(s)
I20260807 04:27:37.003764 23723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:37.007465 23791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:37.011107 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23723
I20260807 04:27:37.011283 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:37.011840 2003 external_mini_cluster.cc:1428] Setting key 4067c673ca3d990653e4632fd1955f25
I20260807 04:27:37.019510 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:37.013627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51927 (local address 127.0.0.1:33369)
0807 04:27:37.014423 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:37.014433 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:37.014473 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:27:37.016157 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:37.016178 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:37.016497 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:37.016820 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:37.018003 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:37.018010 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:37.018022 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:37.018038 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:37.018100 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:37.018133 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:37.018311 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:37.018453 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 04:27:37.018652 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":362,"threads_started":1}
I20260807 04:27:37.020117 23791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9: Bootstrap starting.
I20260807 04:27:37.023067 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:37.023259 23791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:37.024518 23791 log.cc:824] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:37.027696 23791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9: No bootstrap required, opened a new log
I20260807 04:27:37.030833 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:37.028377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38763 (local address 127.0.0.1:33369)
0807 04:27:37.028523 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:37.028527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:37.028869 (+ 342us) server_negotiation.cc:407] Connection header received
0807 04:27:37.029038 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:37.029042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:37.029091 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:37.029168 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:37.030145 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:37.030149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:37.030152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:37.030169 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:37.030208 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:37.030226 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:37.030315 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:37.030453 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 04:27:37.030500 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 04:27:37.036213 23791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03301b9ad61740cb86c5001186f4f6f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33369 } }
I20260807 04:27:37.036542 23791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:37.036623 23791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03301b9ad61740cb86c5001186f4f6f9, State: Initialized, Role: FOLLOWER
I20260807 04:27:37.037099 23791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03301b9ad61740cb86c5001186f4f6f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33369 } }
I20260807 04:27:37.037307 23791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:37.037400 23791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:37.037572 23791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:37.039844 23791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03301b9ad61740cb86c5001186f4f6f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33369 } }
I20260807 04:27:37.040426 23791 leader_election.cc:302] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03301b9ad61740cb86c5001186f4f6f9; no voters:
I20260807 04:27:37.040992 23791 leader_election.cc:288] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:37.041142 23803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:37.041837 23803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [term 1 LEADER]: Becoming Leader. State: Replica: 03301b9ad61740cb86c5001186f4f6f9, State: Running, Role: LEADER
I20260807 04:27:37.042366 23803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03301b9ad61740cb86c5001186f4f6f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33369 } }
I20260807 04:27:37.043008 23791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:37.044993 23804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03301b9ad61740cb86c5001186f4f6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03301b9ad61740cb86c5001186f4f6f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33369 } } }
I20260807 04:27:37.045356 23804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:37.046245 23808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:37.046176 23805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03301b9ad61740cb86c5001186f4f6f9. Latest consensus state: current_term: 1 leader_uuid: "03301b9ad61740cb86c5001186f4f6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03301b9ad61740cb86c5001186f4f6f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33369 } } }
I20260807 04:27:37.046531 23805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:37.054193 23808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:37.063514 23808 catalog_manager.cc:1347] Generated new cluster ID: 7459efe2c7f143ab8cf3d402a9e7741f
I20260807 04:27:37.063648 23808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:37.080154 23808 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:37.080713 23808 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:37.086076 23808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03301b9ad61740cb86c5001186f4f6f9: Generated new TSK 0
I20260807 04:27:37.086814 23808 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:37.091985 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23723
W20260807 04:27:37.093255 23786 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51927
W20260807 04:27:37.094017 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23723: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:37.094164 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23723
2026-08-07T04:27:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1932 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:37 dist-test-slave-zkpd krb5kdc[23824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:37 dist-test-slave-zkpd krb5kdc[23824](info): set up 2 sockets
Aug 07 04:27:37 dist-test-slave-zkpd krb5kdc[23824](info): commencing operation
krb5kdc: starting...
W20260807 04:27:38.235966 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:38 dist-test-slave-zkpd krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:38.541373 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:38.878156 23837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:38.878506 23837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:38.878662 23837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:38.885566 23837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:38.885699 23837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:38.885821 23837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:38.885921 23837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:38.886008 23837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:38.895306 23837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:38.896585 23837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:38.897821 23837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:38.909469 23845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:38.909550 23842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:38.909565 23843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:38.910784 23837 server_base.cc:1034] running on GCE node
I20260807 04:27:38.911584 23837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:38.913004 23837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:38.914206 23837 hybrid_clock.cc:648] HybridClock initialized: now 1786076858914165 us; error 45 us; skew 500 ppm
I20260807 04:27:38.914546 23837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:38 dist-test-slave-zkpd krb5kdc[23824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076858, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:38.917754 23837 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:38.919113 23837 webserver.cc:460] Webserver started at http://127.0.0.1:32867/ using document root <none> and password file <none>
I20260807 04:27:38.919763 23837 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:38.919894 23837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:38.920253 23837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:38.923177 23837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "343b72680c8d4131928a5645183fd7a5"
format_stamp: "Formatted at 2026-08-07 04:27:38 on dist-test-slave-zkpd"
server_key: "487df4ee7b71d02c3153ce81482f30b0"
server_key_iv: "b515cf07ffd30311030d0be94c9854f2"
server_key_version: "encryptionkey@0"
I20260807 04:27:38.924049 23837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "343b72680c8d4131928a5645183fd7a5"
format_stamp: "Formatted at 2026-08-07 04:27:38 on dist-test-slave-zkpd"
server_key: "487df4ee7b71d02c3153ce81482f30b0"
server_key_iv: "b515cf07ffd30311030d0be94c9854f2"
server_key_version: "encryptionkey@0"
I20260807 04:27:38.929082 23837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:27:38.932400 23853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:38.933677 23837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:27:38.933877 23837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "343b72680c8d4131928a5645183fd7a5"
format_stamp: "Formatted at 2026-08-07 04:27:38 on dist-test-slave-zkpd"
server_key: "487df4ee7b71d02c3153ce81482f30b0"
server_key_iv: "b515cf07ffd30311030d0be94c9854f2"
server_key_version: "encryptionkey@0"
I20260807 04:27:38.934048 23837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:38.949826 23837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:38.954041 23837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:38.954329 23837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:38.972610 23837 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34457
I20260807 04:27:38.972651 23904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34457 every 8 connection(s)
I20260807 04:27:38.973925 23837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:38.977464 23905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:38.979923 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23837
I20260807 04:27:38.980115 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:38.980664 2003 external_mini_cluster.cc:1428] Setting key 6257dec4515bfa061b79e4ab62051a9a
I20260807 04:27:38.989739 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5: Bootstrap starting.
I20260807 04:27:38.993469 23905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:38.994776 23905 log.cc:824] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:27:38 dist-test-slave-zkpd krb5kdc[23824](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:27:38.997771 23905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5: No bootstrap required, opened a new log
I20260807 04:27:39.004047 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:38.982428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:48891 (local address 127.0.0.1:34457)
0807 04:27:38.983247 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:38.983260 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:38.983297 (+ 37us) server_negotiation.cc:407] Connection header received
0807 04:27:38.985069 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:38.985092 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:38.985467 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:38.985778 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:38.987548 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:38.988901 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:38.990669 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:38.991312 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:38.991444 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:38.996584 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:38.996591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:38.996605 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:38.996636 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:38.999748 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:39.000674 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:39.000680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:39.000698 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:39.000853 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:39.001333 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:39.001337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:39.001339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:39.001874 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:39.002121 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:39.002649 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 04:27:39.002966 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":380,"threads_started":1}
I20260807 04:27:39.007174 23905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343b72680c8d4131928a5645183fd7a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34457 } }
I20260807 04:27:39.007565 23905 raft_consensus.cc:374] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:39.007673 23905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 343b72680c8d4131928a5645183fd7a5, State: Initialized, Role: FOLLOWER
I20260807 04:27:39.007997 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:39.008318 23905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343b72680c8d4131928a5645183fd7a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34457 } }
I20260807 04:27:39.008551 23905 raft_consensus.cc:388] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:39.008690 23905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:39.008857 23905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:39.010890 23905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343b72680c8d4131928a5645183fd7a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34457 } }
I20260807 04:27:39.011351 23905 leader_election.cc:302] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 343b72680c8d4131928a5645183fd7a5; no voters:
I20260807 04:27:39.011967 23905 leader_election.cc:288] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:39.012135 23914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:39.013558 23914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [term 1 LEADER]: Becoming Leader. State: Replica: 343b72680c8d4131928a5645183fd7a5, State: Running, Role: LEADER
I20260807 04:27:39.014108 23914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343b72680c8d4131928a5645183fd7a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34457 } }
I20260807 04:27:39.014662 23905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:39.017117 23915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "343b72680c8d4131928a5645183fd7a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343b72680c8d4131928a5645183fd7a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34457 } } }
I20260807 04:27:39.017517 23915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:39.017104 23916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 343b72680c8d4131928a5645183fd7a5. Latest consensus state: current_term: 1 leader_uuid: "343b72680c8d4131928a5645183fd7a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "343b72680c8d4131928a5645183fd7a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34457 } } }
I20260807 04:27:39.018059 23916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:39.020092 23920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:39.026762 23920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:39.033941 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:39.012451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:41571 (local address 127.0.0.1:34457)
0807 04:27:39.012633 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:39.012641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:39.014657 (+ 2016us) server_negotiation.cc:407] Connection header received
0807 04:27:39.014943 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:39.014948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:39.015018 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:39.015123 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:39.018791 (+ 3668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:39.019774 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:39.021794 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:39.022361 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:39.022499 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:39.026063 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:39.026069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:39.026075 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:39.026113 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:39.030152 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:39.031105 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:39.031111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:39.031115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:39.031265 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:39.031872 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:39.031877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:39.031881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:39.032377 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:39.032652 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:39.033362 (+ 710us) server_negotiation.cc:299] Negotiation successful
0807 04:27:39.033545 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:27:39.041247 23920 catalog_manager.cc:1347] Generated new cluster ID: 755d8d6aef8445efa6a20d23429a78ee
I20260807 04:27:39.041376 23920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:39.058707 23920 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:39.059952 23920 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:39.066278 23920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 343b72680c8d4131928a5645183fd7a5: Generated new TSK 0
I20260807 04:27:39.067040 23920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:39.073067 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23837
I20260807 04:27:39.074301 23900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:39.162533 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23837
W20260807 04:27:39.182320 23822 connection.cc:535] client connection to 127.0.0.1:34457 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34457): Connection reset by peer (error 104)
2026-08-07T04:27:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2067 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:39 dist-test-slave-zkpd krb5kdc[23938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:39 dist-test-slave-zkpd krb5kdc[23938](info): set up 2 sockets
Aug 07 04:27:39 dist-test-slave-zkpd krb5kdc[23938](info): commencing operation
krb5kdc: starting...
W20260807 04:27:40.283809 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:40 dist-test-slave-zkpd krb5kdc[23938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:40.578758 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:40.911345 23952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:40.911657 23952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:40.911828 23952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:40.918967 23952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:40.919133 23952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:40.919247 23952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:40.919363 23952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:40.919442 23952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:40.928797 23952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:40.930007 23952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:40.931258 23952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:40.942510 23957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:40.942472 23958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:40.942438 23960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:40.943347 23952 server_base.cc:1034] running on GCE node
I20260807 04:27:40.944177 23952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:40.945509 23952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:40.946700 23952 hybrid_clock.cc:648] HybridClock initialized: now 1786076860946645 us; error 43 us; skew 500 ppm
I20260807 04:27:40.947026 23952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:40 dist-test-slave-zkpd krb5kdc[23938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:40.950176 23952 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:40.951525 23952 webserver.cc:460] Webserver started at http://127.0.0.1:35679/ using document root <none> and password file <none>
I20260807 04:27:40.952255 23952 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:40.952390 23952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:40.952692 23952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:40.955739 23952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "4c98518d44c2402d80233528c471eb08"
format_stamp: "Formatted at 2026-08-07 04:27:40 on dist-test-slave-zkpd"
server_key: "e2a34f54b73c558b469b90179271ccfb"
server_key_iv: "5352db763adddeee86b05c194d039cee"
server_key_version: "encryptionkey@0"
I20260807 04:27:40.956568 23952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "4c98518d44c2402d80233528c471eb08"
format_stamp: "Formatted at 2026-08-07 04:27:40 on dist-test-slave-zkpd"
server_key: "e2a34f54b73c558b469b90179271ccfb"
server_key_iv: "5352db763adddeee86b05c194d039cee"
server_key_version: "encryptionkey@0"
I20260807 04:27:40.961447 23952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:27:40.964664 23967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:40.965903 23952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:27:40.966084 23952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "4c98518d44c2402d80233528c471eb08"
format_stamp: "Formatted at 2026-08-07 04:27:40 on dist-test-slave-zkpd"
server_key: "e2a34f54b73c558b469b90179271ccfb"
server_key_iv: "5352db763adddeee86b05c194d039cee"
server_key_version: "encryptionkey@0"
I20260807 04:27:40.966276 23952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:40.986994 23952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:40.991313 23952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:40.991622 23952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:41.009930 23952 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40295
I20260807 04:27:41.009969 24020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40295 every 8 connection(s)
I20260807 04:27:41.011246 23952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:41.014804 24021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:41.016304 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 23952
I20260807 04:27:41.016546 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:41.017140 2003 external_mini_cluster.cc:1428] Setting key c889657e9d167fa16cb1ba3db85be6d1
I20260807 04:27:41.029796 24021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08: Bootstrap starting.
I20260807 04:27:41.031749 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:41.018926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:48345 (local address 127.0.0.1:40295)
0807 04:27:41.019723 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:41.019740 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:41.019790 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:27:41.021703 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:41.021748 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:41.022269 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:41.022718 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:41.023220 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:41.025095 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:41.026513 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:41.027614 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:41.027782 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:41.029021 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:41.029033 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:41.029050 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:41.029752 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:41.029863 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:41.029899 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:41.030056 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:41.030303 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 04:27:41.030477 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":347,"threads_started":1}
I20260807 04:27:41.033643 24021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:41.034976 24021 log.cc:824] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:41.035867 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:41.037784 24021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08: No bootstrap required, opened a new log
I20260807 04:27:41.046975 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:41.040331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:53061 (local address 127.0.0.1:40295)
0807 04:27:41.040529 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:41.040535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:41.040935 (+ 400us) server_negotiation.cc:407] Connection header received
0807 04:27:41.041095 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:41.041105 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:41.041179 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:41.041288 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:41.041730 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:41.042892 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:41.044237 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:41.044987 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:41.045126 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:41.046204 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:41.046211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:41.046216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:41.046238 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:41.046281 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:41.046301 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:41.046417 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:41.046577 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 04:27:41.046627 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:27:41.047155 24021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c98518d44c2402d80233528c471eb08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40295 } }
I20260807 04:27:41.047631 24021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:41.047763 24021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c98518d44c2402d80233528c471eb08, State: Initialized, Role: FOLLOWER
I20260807 04:27:41.048406 24021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c98518d44c2402d80233528c471eb08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40295 } }
I20260807 04:27:41.048699 24021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:41.048848 24021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:41.049022 24021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:41.051435 24021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c98518d44c2402d80233528c471eb08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40295 } }
I20260807 04:27:41.052033 24021 leader_election.cc:302] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c98518d44c2402d80233528c471eb08; no voters:
I20260807 04:27:41.052625 24021 leader_election.cc:288] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:41.052752 24031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:41.053447 24031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [term 1 LEADER]: Becoming Leader. State: Replica: 4c98518d44c2402d80233528c471eb08, State: Running, Role: LEADER
I20260807 04:27:41.054031 24031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c98518d44c2402d80233528c471eb08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40295 } }
I20260807 04:27:41.054761 24021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:41.057080 24032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c98518d44c2402d80233528c471eb08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c98518d44c2402d80233528c471eb08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40295 } } }
I20260807 04:27:41.057657 24032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:41.060094 24033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c98518d44c2402d80233528c471eb08. Latest consensus state: current_term: 1 leader_uuid: "4c98518d44c2402d80233528c471eb08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c98518d44c2402d80233528c471eb08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40295 } } }
I20260807 04:27:41.060361 24033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:41.060899 24040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:41.066124 24040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:41.075369 24040 catalog_manager.cc:1347] Generated new cluster ID: c48cb9716bc74324b7d862d308ec0afe
I20260807 04:27:41.075498 24040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:41.093026 24040 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:41.094238 24040 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:41.103030 24040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c98518d44c2402d80233528c471eb08: Generated new TSK 0
I20260807 04:27:41.104092 24040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:41.120471 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23952
W20260807 04:27:41.121536 24016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.39:48345
W20260807 04:27:41.122252 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 23952: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:41.122413 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 23952
W20260807 04:27:41.140100 23936 connection.cc:535] client connection to 127.0.0.1:40295 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40295): Connection reset by peer (error 104)
2026-08-07T04:27:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1957 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:41 dist-test-slave-zkpd krb5kdc[24052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:41 dist-test-slave-zkpd krb5kdc[24052](info): set up 2 sockets
Aug 07 04:27:41 dist-test-slave-zkpd krb5kdc[24052](info): commencing operation
krb5kdc: starting...
W20260807 04:27:42.259199 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:42 dist-test-slave-zkpd krb5kdc[24052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:42.557303 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:42.891332 24065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:42.891607 24065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:42.891697 24065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:42.898797 24065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:42.898896 24065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:42.898949 24065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:42.899040 24065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:42.899106 24065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:42.908936 24065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:42.910157 24065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:42.911427 24065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:42.923064 24074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:42.923127 24071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:42.923101 24072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:42.924167 24065 server_base.cc:1034] running on GCE node
I20260807 04:27:42.924834 24065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:42.926059 24065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:42.927321 24065 hybrid_clock.cc:648] HybridClock initialized: now 1786076862927295 us; error 57 us; skew 500 ppm
I20260807 04:27:42.927670 24065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:42 dist-test-slave-zkpd krb5kdc[24052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:42.930711 24065 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:42.932233 24065 webserver.cc:460] Webserver started at http://127.0.0.1:44875/ using document root <none> and password file <none>
I20260807 04:27:42.932894 24065 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:42.933038 24065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:42.933339 24065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:42.936350 24065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "88d685149a7d4d788ba4b26aca8592b4"
format_stamp: "Formatted at 2026-08-07 04:27:42 on dist-test-slave-zkpd"
server_key: "f7158d73cd33ecd50757405e050e1e6b"
server_key_iv: "5e83e7bf825095848f8a00ea5b292ab0"
server_key_version: "encryptionkey@0"
I20260807 04:27:42.937136 24065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "88d685149a7d4d788ba4b26aca8592b4"
format_stamp: "Formatted at 2026-08-07 04:27:42 on dist-test-slave-zkpd"
server_key: "f7158d73cd33ecd50757405e050e1e6b"
server_key_iv: "5e83e7bf825095848f8a00ea5b292ab0"
server_key_version: "encryptionkey@0"
I20260807 04:27:42.942163 24065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 04:27:42.945462 24081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:42.946610 24065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 04:27:42.946791 24065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "88d685149a7d4d788ba4b26aca8592b4"
format_stamp: "Formatted at 2026-08-07 04:27:42 on dist-test-slave-zkpd"
server_key: "f7158d73cd33ecd50757405e050e1e6b"
server_key_iv: "5e83e7bf825095848f8a00ea5b292ab0"
server_key_version: "encryptionkey@0"
I20260807 04:27:42.946985 24065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:42.952623 24065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:42.956897 24065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:42.957183 24065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:42.974705 24065 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33771
I20260807 04:27:42.974789 24133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33771 every 8 connection(s)
I20260807 04:27:42.976039 24065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:42.979750 24134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:42.983489 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24065
I20260807 04:27:42.983672 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:42.984201 2003 external_mini_cluster.cc:1428] Setting key dd3fa759e719c6ff2d7d6a742f243441
I20260807 04:27:42.991816 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:42.985979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:45501 (local address 127.0.0.1:33771)
0807 04:27:42.986782 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:42.986793 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:42.986836 (+ 43us) server_negotiation.cc:407] Connection header received
0807 04:27:42.988487 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:42.988508 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:42.988874 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:42.989191 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:42.990277 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:42.990285 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:42.990298 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:42.990313 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:42.990376 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:42.990396 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:42.990567 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:42.990706 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 04:27:42.990948 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":367,"threads_started":1}
I20260807 04:27:42.992671 24134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4: Bootstrap starting.
I20260807 04:27:42.995651 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:42.996158 24134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:42.997149 24134 log.cc:824] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:43.000308 24134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4: No bootstrap required, opened a new log
I20260807 04:27:43.002525 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:43.000056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:54725 (local address 127.0.0.1:33771)
0807 04:27:43.000223 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:43.000229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:43.000716 (+ 487us) server_negotiation.cc:407] Connection header received
0807 04:27:43.000823 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:43.000828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:43.000886 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:43.001002 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:43.001889 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:43.001896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:43.001915 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:43.001935 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:43.001989 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:43.002007 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:43.002130 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:43.002180 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 04:27:43.002242 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:27:43.008322 24134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d685149a7d4d788ba4b26aca8592b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260807 04:27:43.008689 24134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:43.008805 24134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d685149a7d4d788ba4b26aca8592b4, State: Initialized, Role: FOLLOWER
I20260807 04:27:43.009397 24134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d685149a7d4d788ba4b26aca8592b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260807 04:27:43.009660 24134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:43.009806 24134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:43.009999 24134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:43.012226 24134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d685149a7d4d788ba4b26aca8592b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260807 04:27:43.012720 24134 leader_election.cc:302] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d685149a7d4d788ba4b26aca8592b4; no voters:
I20260807 04:27:43.013244 24134 leader_election.cc:288] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:43.013397 24145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:43.014071 24145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [term 1 LEADER]: Becoming Leader. State: Replica: 88d685149a7d4d788ba4b26aca8592b4, State: Running, Role: LEADER
I20260807 04:27:43.014631 24145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d685149a7d4d788ba4b26aca8592b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } }
I20260807 04:27:43.015259 24134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:43.017844 24146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d685149a7d4d788ba4b26aca8592b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d685149a7d4d788ba4b26aca8592b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } } }
I20260807 04:27:43.018240 24146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:43.019080 24147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d685149a7d4d788ba4b26aca8592b4. Latest consensus state: current_term: 1 leader_uuid: "88d685149a7d4d788ba4b26aca8592b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d685149a7d4d788ba4b26aca8592b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33771 } } }
I20260807 04:27:43.019434 24147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:43.020054 24152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:43.028959 24152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:43.040966 24152 catalog_manager.cc:1347] Generated new cluster ID: 9643de75391d4ff58a24ca7ffe7d256d
I20260807 04:27:43.041076 24152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:43.062829 24152 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:43.063256 24152 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:43.076614 24152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d685149a7d4d788ba4b26aca8592b4: Generated new TSK 0
I20260807 04:27:43.077368 24152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:43.081869 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24065
W20260807 04:27:43.083215 24129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.39:45501
W20260807 04:27:43.083948 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24065: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:43.084105 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24065
2026-08-07T04:27:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1964 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:43 dist-test-slave-zkpd krb5kdc[24167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:43 dist-test-slave-zkpd krb5kdc[24167](info): set up 2 sockets
Aug 07 04:27:43 dist-test-slave-zkpd krb5kdc[24167](info): commencing operation
krb5kdc: starting...
W20260807 04:27:44.223042 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:44 dist-test-slave-zkpd krb5kdc[24167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:44.509006 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:44.856810 24180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:44.857095 24180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:44.857307 24180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:44.865003 24180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:44.865132 24180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:44.865221 24180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:44.865327 24180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:44.865413 24180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:44.874997 24180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:44.876259 24180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:44.877599 24180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:44.889526 24185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:44.889480 24188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:44.889566 24186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:44.890471 24180 server_base.cc:1034] running on GCE node
I20260807 04:27:44.891235 24180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:44.892593 24180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:44.893842 24180 hybrid_clock.cc:648] HybridClock initialized: now 1786076864893803 us; error 42 us; skew 500 ppm
I20260807 04:27:44.894183 24180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:44 dist-test-slave-zkpd krb5kdc[24167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:44.901561 24180 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:44.903108 24180 webserver.cc:460] Webserver started at http://127.0.0.1:33517/ using document root <none> and password file <none>
I20260807 04:27:44.903761 24180 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:44.903867 24180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:44.904214 24180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:44.907069 24180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "fa221a46fa1f44cea8969205d5a98697"
format_stamp: "Formatted at 2026-08-07 04:27:44 on dist-test-slave-zkpd"
server_key: "2661d2ab298982304624c9eb3138494b"
server_key_iv: "142955eda8d1160cf041df732f45067c"
server_key_version: "encryptionkey@0"
I20260807 04:27:44.907769 24180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "fa221a46fa1f44cea8969205d5a98697"
format_stamp: "Formatted at 2026-08-07 04:27:44 on dist-test-slave-zkpd"
server_key: "2661d2ab298982304624c9eb3138494b"
server_key_iv: "142955eda8d1160cf041df732f45067c"
server_key_version: "encryptionkey@0"
I20260807 04:27:44.912886 24180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 04:27:44.916146 24195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:44.917459 24180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:44.917598 24180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "fa221a46fa1f44cea8969205d5a98697"
format_stamp: "Formatted at 2026-08-07 04:27:44 on dist-test-slave-zkpd"
server_key: "2661d2ab298982304624c9eb3138494b"
server_key_iv: "142955eda8d1160cf041df732f45067c"
server_key_version: "encryptionkey@0"
I20260807 04:27:44.917747 24180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:44.936647 24180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:44.940339 24180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:44.940552 24180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:44.958750 24180 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43667
I20260807 04:27:44.958823 24247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43667 every 8 connection(s)
I20260807 04:27:44.960160 24180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:44.963666 24248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:44.967365 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24180
I20260807 04:27:44.967584 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:44.968155 2003 external_mini_cluster.cc:1428] Setting key 0c4bf88103a3a81a6c0ee3c11b126361
I20260807 04:27:44.976547 24248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697: Bootstrap starting.
I20260807 04:27:44.980392 24248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:44.981833 24248 log.cc:824] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697: Log is configured to *not* fsync() on all Append() calls
Aug 07 04:27:44 dist-test-slave-zkpd krb5kdc[24167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 04:27:44.985227 24248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697: No bootstrap required, opened a new log
I20260807 04:27:44.991654 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:44.969935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:47571 (local address 127.0.0.1:43667)
0807 04:27:44.970661 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:44.970671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:44.970710 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:27:44.972420 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:44.972442 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:44.972838 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:44.973148 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:44.974981 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:44.976312 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:44.978112 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:44.978767 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:44.978898 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:44.984238 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:44.984246 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:44.984261 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:44.984299 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:44.987368 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:44.988300 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:44.988305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:44.988310 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:44.988527 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:44.989023 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:44.989026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:44.989028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:44.989514 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:44.989746 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:44.990250 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 04:27:44.990572 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":322,"threads_started":1}
I20260807 04:27:44.994807 24248 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa221a46fa1f44cea8969205d5a98697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 04:27:44.995252 24248 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:44.995373 24248 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa221a46fa1f44cea8969205d5a98697, State: Initialized, Role: FOLLOWER
I20260807 04:27:44.995774 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:44.996057 24248 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa221a46fa1f44cea8969205d5a98697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 04:27:44.996326 24248 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:44.996459 24248 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:44.996616 24248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:44.999871 24248 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa221a46fa1f44cea8969205d5a98697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 04:27:45.000481 24248 leader_election.cc:302] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa221a46fa1f44cea8969205d5a98697; no voters:
I20260807 04:27:45.001428 24248 leader_election.cc:288] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:45.001529 24257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:45.002135 24257 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [term 1 LEADER]: Becoming Leader. State: Replica: fa221a46fa1f44cea8969205d5a98697, State: Running, Role: LEADER
I20260807 04:27:45.002808 24257 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa221a46fa1f44cea8969205d5a98697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } }
I20260807 04:27:45.005147 24257 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa221a46fa1f44cea8969205d5a98697. Latest consensus state: current_term: 1 leader_uuid: "fa221a46fa1f44cea8969205d5a98697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa221a46fa1f44cea8969205d5a98697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } } }
I20260807 04:27:45.005496 24257 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:45.006747 24260 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa221a46fa1f44cea8969205d5a98697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa221a46fa1f44cea8969205d5a98697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43667 } } }
I20260807 04:27:45.007081 24260 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:45.010174 24248 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:45.014623 24262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:45.019325 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:45.001088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:37361 (local address 127.0.0.1:43667)
0807 04:27:45.001288 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:45.001296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:45.001999 (+ 703us) server_negotiation.cc:407] Connection header received
0807 04:27:45.002282 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:45.002287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:45.002369 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:45.002435 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:45.004504 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:45.005724 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:45.007526 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:45.008249 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:45.008376 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:45.011166 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:45.011171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:45.011175 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:45.011207 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:45.014926 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:45.015704 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:45.015709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:45.015713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:45.015859 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:45.016599 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:45.016604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:45.016608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:45.017191 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:45.017487 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:45.018723 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0807 04:27:45.018934 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 04:27:45.021647 24262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:45.032218 24262 catalog_manager.cc:1347] Generated new cluster ID: 0ffef62f7db0442bbb5198212a87dc09
I20260807 04:27:45.032332 24262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:45.048394 24262 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:45.049597 24262 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:45.054464 24262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa221a46fa1f44cea8969205d5a98697: Generated new TSK 0
I20260807 04:27:45.055184 24262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:45.061451 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24180
I20260807 04:27:45.062564 24243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:45.149843 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24180
W20260807 04:27:45.169795 24165 connection.cc:535] client connection to 127.0.0.1:43667 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43667): Connection reset by peer (error 104)
2026-08-07T04:27:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2069 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:45 dist-test-slave-zkpd krb5kdc[24281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:45 dist-test-slave-zkpd krb5kdc[24281](info): set up 2 sockets
Aug 07 04:27:45 dist-test-slave-zkpd krb5kdc[24281](info): commencing operation
krb5kdc: starting...
W20260807 04:27:46.287051 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:46 dist-test-slave-zkpd krb5kdc[24281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:46.566488 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43337
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:46.898092 24296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:46.898345 24296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:46.898440 24296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:46.905267 24296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:46.905400 24296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:46.905509 24296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:46.905613 24296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:46.905699 24296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:46.914978 24296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43337
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:46.916164 24296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:46.917449 24296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:46.929236 24301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:46.929193 24302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:46.929282 24304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:46.930047 24296 server_base.cc:1034] running on GCE node
I20260807 04:27:46.930763 24296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:46.932206 24296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:46.933435 24296 hybrid_clock.cc:648] HybridClock initialized: now 1786076866933404 us; error 40 us; skew 500 ppm
I20260807 04:27:46.933833 24296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:46 dist-test-slave-zkpd krb5kdc[24281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:46.937032 24296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:46.938421 24296 webserver.cc:460] Webserver started at http://127.0.0.1:46525/ using document root <none> and password file <none>
I20260807 04:27:46.939078 24296 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:46.939240 24296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:46.939563 24296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:46.942509 24296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "1a42fd990b3c4e16a4e311af91337849"
format_stamp: "Formatted at 2026-08-07 04:27:46 on dist-test-slave-zkpd"
server_key: "04b5a6a5ae02e2dca439fa96f2a5aa06"
server_key_iv: "3c983b4f2a99bbf9a591720882ed222b"
server_key_version: "encryptionkey@0"
I20260807 04:27:46.943259 24296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "1a42fd990b3c4e16a4e311af91337849"
format_stamp: "Formatted at 2026-08-07 04:27:46 on dist-test-slave-zkpd"
server_key: "04b5a6a5ae02e2dca439fa96f2a5aa06"
server_key_iv: "3c983b4f2a99bbf9a591720882ed222b"
server_key_version: "encryptionkey@0"
I20260807 04:27:46.948418 24296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 04:27:46.951813 24312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:46.953102 24296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 04:27:46.953279 24296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "1a42fd990b3c4e16a4e311af91337849"
format_stamp: "Formatted at 2026-08-07 04:27:46 on dist-test-slave-zkpd"
server_key: "04b5a6a5ae02e2dca439fa96f2a5aa06"
server_key_iv: "3c983b4f2a99bbf9a591720882ed222b"
server_key_version: "encryptionkey@0"
I20260807 04:27:46.953470 24296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:46.978843 24296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:46.983157 24296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:46.983453 24296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:47.001868 24296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43337
I20260807 04:27:47.001901 24363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43337 every 8 connection(s)
I20260807 04:27:47.003204 24296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:47.005167 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24296
I20260807 04:27:47.005359 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:47.005910 2003 external_mini_cluster.cc:1428] Setting key 2e9f8c8f8428c8f68e13d0bcd88f802c
I20260807 04:27:47.007589 24365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:47.019212 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:47.007723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:54241 (local address 127.0.0.1:43337)
0807 04:27:47.008572 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:47.008583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:47.008622 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:27:47.010230 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:47.010251 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:47.010624 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:47.010951 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:47.011532 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:47.012939 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:47.014849 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:47.015495 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:47.015614 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:47.016978 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:47.016986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:47.017002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:47.017527 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:47.017598 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:47.017621 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:47.017741 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:47.018062 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 04:27:47.018268 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":416,"threads_started":1}
I20260807 04:27:47.020646 24365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849: Bootstrap starting.
I20260807 04:27:47.023051 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:47.023864 24365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:47.024891 24365 log.cc:824] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:47.028064 24365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849: No bootstrap required, opened a new log
I20260807 04:27:47.034252 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:47.027620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:41661 (local address 127.0.0.1:43337)
0807 04:27:47.027834 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:47.027841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:47.028257 (+ 416us) server_negotiation.cc:407] Connection header received
0807 04:27:47.028407 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:47.028412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:47.028482 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:47.028607 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:47.029039 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:47.030198 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:47.031467 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:47.032210 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:47.032353 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:47.033408 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:47.033415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:47.033419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:47.033441 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:47.033499 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:47.033518 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:47.033636 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:47.033841 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 04:27:47.033892 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 04:27:47.037340 24365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a42fd990b3c4e16a4e311af91337849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43337 } }
I20260807 04:27:47.037667 24365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:47.037743 24365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a42fd990b3c4e16a4e311af91337849, State: Initialized, Role: FOLLOWER
I20260807 04:27:47.038216 24365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a42fd990b3c4e16a4e311af91337849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43337 } }
I20260807 04:27:47.038406 24365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:47.038504 24365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:47.038621 24365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:47.040988 24365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a42fd990b3c4e16a4e311af91337849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43337 } }
I20260807 04:27:47.041420 24365 leader_election.cc:302] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a42fd990b3c4e16a4e311af91337849; no voters:
I20260807 04:27:47.042008 24365 leader_election.cc:288] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:47.042143 24375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:47.042901 24375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [term 1 LEADER]: Becoming Leader. State: Replica: 1a42fd990b3c4e16a4e311af91337849, State: Running, Role: LEADER
I20260807 04:27:47.043540 24375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a42fd990b3c4e16a4e311af91337849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43337 } }
I20260807 04:27:47.044210 24365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:47.046438 24377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a42fd990b3c4e16a4e311af91337849. Latest consensus state: current_term: 1 leader_uuid: "1a42fd990b3c4e16a4e311af91337849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a42fd990b3c4e16a4e311af91337849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43337 } } }
I20260807 04:27:47.046823 24377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:47.046624 24376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a42fd990b3c4e16a4e311af91337849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a42fd990b3c4e16a4e311af91337849" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43337 } } }
I20260807 04:27:47.047108 24376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:47.048987 24384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:47.054080 24384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:47.062909 24384 catalog_manager.cc:1347] Generated new cluster ID: 09a4d521c2464555b9cf5f14b0188fba
I20260807 04:27:47.062995 24384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:47.079371 24384 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:47.080552 24384 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:47.088236 24384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a42fd990b3c4e16a4e311af91337849: Generated new TSK 0
I20260807 04:27:47.089165 24384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:47.101846 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24296
W20260807 04:27:47.102991 24359 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.39:54241
W20260807 04:27:47.103765 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24296: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:47.103931 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24296
W20260807 04:27:47.123746 24279 connection.cc:535] client connection to 127.0.0.1:43337 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43337): Connection reset by peer (error 104)
2026-08-07T04:27:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1949 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:47 dist-test-slave-zkpd krb5kdc[24398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:47 dist-test-slave-zkpd krb5kdc[24398](info): set up 2 sockets
Aug 07 04:27:47 dist-test-slave-zkpd krb5kdc[24398](info): commencing operation
krb5kdc: starting...
W20260807 04:27:48.235672 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:48 dist-test-slave-zkpd krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 04:27:48.529207 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45151
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:48.862356 24411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:48.862619 24411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:48.862710 24411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:48.869695 24411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:48.869788 24411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:48.869834 24411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:48.869894 24411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:48.869936 24411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:48.879164 24411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45151
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:48.880291 24411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:48.881484 24411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:48.892943 24417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:48.892949 24419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:48.892916 24416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:48.893870 24411 server_base.cc:1034] running on GCE node
I20260807 04:27:48.894517 24411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:48.895736 24411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:48.896970 24411 hybrid_clock.cc:648] HybridClock initialized: now 1786076868896924 us; error 54 us; skew 500 ppm
I20260807 04:27:48.897307 24411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:48 dist-test-slave-zkpd krb5kdc[24398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076868, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:48.900281 24411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 04:27:48.901502 24411 webserver.cc:460] Webserver started at http://127.0.0.1:41145/ using document root <none> and password file <none>
I20260807 04:27:48.902083 24411 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:48.902177 24411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:48.902400 24411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:48.905248 24411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "a0feecd2d03e43c393d87c9f361cf3e2"
format_stamp: "Formatted at 2026-08-07 04:27:48 on dist-test-slave-zkpd"
server_key: "8c4b8d067ed8de25bfbd8cd183850d76"
server_key_iv: "29714385fc7efcb1d7228ce59943ccab"
server_key_version: "encryptionkey@0"
I20260807 04:27:48.905947 24411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "a0feecd2d03e43c393d87c9f361cf3e2"
format_stamp: "Formatted at 2026-08-07 04:27:48 on dist-test-slave-zkpd"
server_key: "8c4b8d067ed8de25bfbd8cd183850d76"
server_key_iv: "29714385fc7efcb1d7228ce59943ccab"
server_key_version: "encryptionkey@0"
I20260807 04:27:48.911182 24411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:27:48.914527 24426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:48.915746 24411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:27:48.915882 24411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "a0feecd2d03e43c393d87c9f361cf3e2"
format_stamp: "Formatted at 2026-08-07 04:27:48 on dist-test-slave-zkpd"
server_key: "8c4b8d067ed8de25bfbd8cd183850d76"
server_key_iv: "29714385fc7efcb1d7228ce59943ccab"
server_key_version: "encryptionkey@0"
I20260807 04:27:48.916155 24411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:48.922012 24411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:48.925580 24411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:48.925827 24411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:48.944288 24411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45151
I20260807 04:27:48.944361 24479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45151 every 8 connection(s)
I20260807 04:27:48.945622 24411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:48.947443 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24411
I20260807 04:27:48.947626 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:48.948140 2003 external_mini_cluster.cc:1428] Setting key a661a72c54f2f40f9597a6fba9af275c
I20260807 04:27:48.949580 24480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:48.955538 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:48.949828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:49091 (local address 127.0.0.1:45151)
0807 04:27:48.950591 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:48.950602 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:48.950642 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:27:48.952284 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:48.952305 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:48.952632 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:48.952942 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:48.954043 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:48.954050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:48.954061 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:48.954075 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:48.954150 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:48.954170 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:48.954339 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:48.954468 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 04:27:48.954667 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":352,"threads_started":1}
I20260807 04:27:48.959246 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:48.962049 24480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2: Bootstrap starting.
I20260807 04:27:48.965298 24480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:48.966094 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:48.963547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:38131 (local address 127.0.0.1:45151)
0807 04:27:48.963717 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:48.963722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:48.964239 (+ 517us) server_negotiation.cc:407] Connection header received
0807 04:27:48.964364 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:48.964369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:48.964424 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:48.964531 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:48.965435 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:48.965441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:48.965447 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:48.965466 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:48.965515 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:48.965536 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:48.965657 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:48.965757 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 04:27:48.965825 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:27:48.966681 24480 log.cc:824] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:48.969472 24480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2: No bootstrap required, opened a new log
I20260807 04:27:48.976426 24480 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260807 04:27:48.976747 24480 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:48.976818 24480 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0feecd2d03e43c393d87c9f361cf3e2, State: Initialized, Role: FOLLOWER
I20260807 04:27:48.977484 24480 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260807 04:27:48.977777 24480 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:48.977934 24480 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:48.978065 24480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:48.980147 24480 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260807 04:27:48.980561 24480 leader_election.cc:302] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0feecd2d03e43c393d87c9f361cf3e2; no voters:
I20260807 04:27:48.981073 24480 leader_election.cc:288] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:48.981210 24491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:48.981974 24491 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [term 1 LEADER]: Becoming Leader. State: Replica: a0feecd2d03e43c393d87c9f361cf3e2, State: Running, Role: LEADER
I20260807 04:27:48.982528 24491 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } }
I20260807 04:27:48.983294 24480 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:48.985491 24492 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } } }
I20260807 04:27:48.985816 24492 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:48.986274 24493 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0feecd2d03e43c393d87c9f361cf3e2. Latest consensus state: current_term: 1 leader_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0feecd2d03e43c393d87c9f361cf3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45151 } } }
I20260807 04:27:48.986568 24493 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:48.987299 24499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:48.992349 24499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:49.001801 24499 catalog_manager.cc:1347] Generated new cluster ID: f681389b31d14f84875cf736f5e42a8c
I20260807 04:27:49.001935 24499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:49.014139 24499 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:49.014599 24499 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:49.023392 24499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0feecd2d03e43c393d87c9f361cf3e2: Generated new TSK 0
I20260807 04:27:49.024312 24499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:49.028467 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24411
W20260807 04:27:49.029690 24475 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.39:49091
W20260807 04:27:49.030475 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24411: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:49.030627 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24411
2026-08-07T04:27:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1927 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:49 dist-test-slave-zkpd krb5kdc[24513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:49 dist-test-slave-zkpd krb5kdc[24513](info): set up 2 sockets
Aug 07 04:27:49 dist-test-slave-zkpd krb5kdc[24513](info): commencing operation
krb5kdc: starting...
W20260807 04:27:50.163000 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:50 dist-test-slave-zkpd krb5kdc[24513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:50.465116 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36741
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:36741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:50.803721 24526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:50.804095 24526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:50.804219 24526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:50.811213 24526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:50.811314 24526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:50.811359 24526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:50.811420 24526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:50.811456 24526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:50.821201 24526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:36741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:36741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:50.822350 24526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:50.823570 24526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:50.834975 24531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:50.834975 24532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:50.835003 24534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:50.836288 24526 server_base.cc:1034] running on GCE node
I20260807 04:27:50.837074 24526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:50.838395 24526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:50.839632 24526 hybrid_clock.cc:648] HybridClock initialized: now 1786076870839589 us; error 46 us; skew 500 ppm
I20260807 04:27:50.840039 24526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:50 dist-test-slave-zkpd krb5kdc[24513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076870, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:50.847697 24526 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:50.849262 24526 webserver.cc:460] Webserver started at http://127.1.244.254:33745/ using document root <none> and password file <none>
I20260807 04:27:50.849941 24526 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:50.850114 24526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:50.850432 24526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:50.853440 24526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "2403bb87f8ea42d484f58e57908801a7"
format_stamp: "Formatted at 2026-08-07 04:27:50 on dist-test-slave-zkpd"
server_key: "b7d5aeb59fb815d63d275d4c78cb1444"
server_key_iv: "a17371bef75bad3620bb23bdc95573b0"
server_key_version: "encryptionkey@0"
I20260807 04:27:50.854254 24526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "2403bb87f8ea42d484f58e57908801a7"
format_stamp: "Formatted at 2026-08-07 04:27:50 on dist-test-slave-zkpd"
server_key: "b7d5aeb59fb815d63d275d4c78cb1444"
server_key_iv: "a17371bef75bad3620bb23bdc95573b0"
server_key_version: "encryptionkey@0"
I20260807 04:27:50.859370 24526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 04:27:50.862749 24541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:50.863988 24526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:27:50.864179 24526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "2403bb87f8ea42d484f58e57908801a7"
format_stamp: "Formatted at 2026-08-07 04:27:50 on dist-test-slave-zkpd"
server_key: "b7d5aeb59fb815d63d275d4c78cb1444"
server_key_iv: "a17371bef75bad3620bb23bdc95573b0"
server_key_version: "encryptionkey@0"
I20260807 04:27:50.864375 24526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:50.889153 24526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:50.892841 24526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:50.893127 24526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:50.911993 24526 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:36741
I20260807 04:27:50.912031 24594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:36741 every 8 connection(s)
I20260807 04:27:50.913340 24526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:50.916004 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24526
I20260807 04:27:50.916215 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:50.916778 2003 external_mini_cluster.cc:1428] Setting key 9dff849fb5923ffc170d776652e13e6e
I20260807 04:27:50.917655 24595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:50.929246 24595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7: Bootstrap starting.
Aug 07 04:27:50 dist-test-slave-zkpd krb5kdc[24513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:50.932895 24595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:50.933923 24595 log.cc:824] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:50.937218 24595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7: No bootstrap required, opened a new log
I20260807 04:27:50.939554 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:50.918599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53273 (local address 127.1.244.254:36741)
0807 04:27:50.919370 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:50.919382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:50.919423 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:27:50.921063 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:50.921082 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:50.921446 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:50.921851 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:50.923263 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:50.924600 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:50.925958 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:50.926650 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:50.926885 (+ 235us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:50.931574 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:50.931581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:50.931594 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:50.931626 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:50.935184 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:50.936210 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:50.936216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:50.936221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:50.936381 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:50.936941 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:50.936944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:50.936946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:50.937472 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:50.937707 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:50.938245 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 04:27:50.938549 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":364,"threads_started":1}
I20260807 04:27:50.943287 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:50.946172 24595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2403bb87f8ea42d484f58e57908801a7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36741 } }
I20260807 04:27:50.946637 24595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:50.946743 24595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2403bb87f8ea42d484f58e57908801a7, State: Initialized, Role: FOLLOWER
I20260807 04:27:50.947484 24595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2403bb87f8ea42d484f58e57908801a7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36741 } }
I20260807 04:27:50.947749 24595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:50.947914 24595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:50.948097 24595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:50.951489 24595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2403bb87f8ea42d484f58e57908801a7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36741 } }
I20260807 04:27:50.952178 24595 leader_election.cc:302] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2403bb87f8ea42d484f58e57908801a7; no voters:
I20260807 04:27:50.952850 24595 leader_election.cc:288] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:50.952945 24606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:50.954218 24606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [term 1 LEADER]: Becoming Leader. State: Replica: 2403bb87f8ea42d484f58e57908801a7, State: Running, Role: LEADER
I20260807 04:27:50.954892 24606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2403bb87f8ea42d484f58e57908801a7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36741 } }
I20260807 04:27:50.955196 24595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:50.957767 24608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2403bb87f8ea42d484f58e57908801a7. Latest consensus state: current_term: 1 leader_uuid: "2403bb87f8ea42d484f58e57908801a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2403bb87f8ea42d484f58e57908801a7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36741 } } }
I20260807 04:27:50.958179 24608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:50.958789 24607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2403bb87f8ea42d484f58e57908801a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2403bb87f8ea42d484f58e57908801a7" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 36741 } } }
I20260807 04:27:50.959128 24607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:50.962276 24614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:50.966782 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:50.948698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35987 (local address 127.1.244.254:36741)
0807 04:27:50.948889 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:50.948894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:50.949414 (+ 520us) server_negotiation.cc:407] Connection header received
0807 04:27:50.949587 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:50.949590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:50.949645 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:50.949727 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:50.951596 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:50.952590 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:50.954354 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:50.954828 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:50.954965 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:50.958529 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:50.958538 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:50.958546 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:50.958589 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:50.962892 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:50.963998 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:50.964004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:50.964008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:50.964163 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:50.964747 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:50.964753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:50.964760 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:50.965179 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:50.965461 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:50.966163 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 04:27:50.966367 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:27:50.968123 24614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:50.976655 24614 catalog_manager.cc:1347] Generated new cluster ID: c03746a568ee4ff386a4fa08c22ec178
I20260807 04:27:50.976758 24614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:50.988391 24614 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:50.989638 24614 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:50.995698 24614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2403bb87f8ea42d484f58e57908801a7: Generated new TSK 0
I20260807 04:27:50.996448 24614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:51.003266 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24526
I20260807 04:27:51.004453 24590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:51.093947 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24526
W20260807 04:27:51.113960 24511 connection.cc:535] client connection to 127.1.244.254:36741 recv error: Network error: failed to read from TLS socket (remote: 127.1.244.254:36741): Connection reset by peer (error 104)
2026-08-07T04:27:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2062 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:51 dist-test-slave-zkpd krb5kdc[24628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:51 dist-test-slave-zkpd krb5kdc[24628](info): set up 2 sockets
Aug 07 04:27:51 dist-test-slave-zkpd krb5kdc[24628](info): commencing operation
krb5kdc: starting...
W20260807 04:27:52.223534 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.008s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:52 dist-test-slave-zkpd krb5kdc[24628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:52.512506 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40849
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:52.844626 24641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:52.844960 24641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:52.845126 24641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:52.852141 24641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:52.852274 24641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:52.852378 24641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:52.852489 24641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:52.852563 24641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:52.861868 24641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40849
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:52.863072 24641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:52.864423 24641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:52.876121 24647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:52.876181 24649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:52.876217 24646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:52.877014 24641 server_base.cc:1034] running on GCE node
I20260807 04:27:52.877734 24641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:52.879096 24641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:52.880290 24641 hybrid_clock.cc:648] HybridClock initialized: now 1786076872880250 us; error 40 us; skew 500 ppm
I20260807 04:27:52.880630 24641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:52 dist-test-slave-zkpd krb5kdc[24628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076872, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:52.883594 24641 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:52.885110 24641 webserver.cc:460] Webserver started at http://127.1.244.254:40729/ using document root <none> and password file <none>
I20260807 04:27:52.885762 24641 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:52.885910 24641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:52.886204 24641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:52.889134 24641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66"
format_stamp: "Formatted at 2026-08-07 04:27:52 on dist-test-slave-zkpd"
server_key: "b9a9ac1b552cbaafa50135dd7caa7511"
server_key_iv: "97d9fa143df59a3874c535f4c2d3d166"
server_key_version: "encryptionkey@0"
I20260807 04:27:52.889895 24641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66"
format_stamp: "Formatted at 2026-08-07 04:27:52 on dist-test-slave-zkpd"
server_key: "b9a9ac1b552cbaafa50135dd7caa7511"
server_key_iv: "97d9fa143df59a3874c535f4c2d3d166"
server_key_version: "encryptionkey@0"
I20260807 04:27:52.894767 24641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:27:52.898142 24656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:52.899318 24641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 04:27:52.899525 24641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66"
format_stamp: "Formatted at 2026-08-07 04:27:52 on dist-test-slave-zkpd"
server_key: "b9a9ac1b552cbaafa50135dd7caa7511"
server_key_iv: "97d9fa143df59a3874c535f4c2d3d166"
server_key_version: "encryptionkey@0"
I20260807 04:27:52.899712 24641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:52.931277 24641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:52.935137 24641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:52.935436 24641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:52.953864 24641 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40849
I20260807 04:27:52.953902 24708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40849 every 8 connection(s)
I20260807 04:27:52.955160 24641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:52.958786 24709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:52.962194 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24641
I20260807 04:27:52.962424 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:52.963009 2003 external_mini_cluster.cc:1428] Setting key 938386317f0690858f2b1ff756805f3b
I20260807 04:27:52.970582 24709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66: Bootstrap starting.
I20260807 04:27:52.973699 24709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:52.974805 24709 log.cc:824] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:52.975697 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:52.964880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33179 (local address 127.1.244.254:40849)
0807 04:27:52.965570 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:52.965581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:52.965619 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:27:52.967279 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:52.967301 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:52.967660 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:52.968078 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:52.968612 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:52.970003 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:52.971296 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:52.972005 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:52.972195 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:52.973470 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:52.973478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:52.973491 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:52.974061 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:52.974134 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:52.974161 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:52.974328 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:52.974615 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 04:27:52.974761 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":292,"threads_started":1}
I20260807 04:27:52.977873 24709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66: No bootstrap required, opened a new log
I20260807 04:27:52.979622 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:52.986515 24709 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40849 } }
I20260807 04:27:52.986855 24709 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:52.986954 24709 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef40d76c8e1b46d0b8d7c466f56c5d66, State: Initialized, Role: FOLLOWER
I20260807 04:27:52.987437 24709 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40849 } }
I20260807 04:27:52.987665 24709 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:52.987815 24709 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:52.988031 24709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:52.990154 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:52.983667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60987 (local address 127.1.244.254:40849)
0807 04:27:52.983826 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:52.983831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:52.984353 (+ 522us) server_negotiation.cc:407] Connection header received
0807 04:27:52.984474 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:52.984480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:52.984551 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:52.984668 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:52.985167 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:52.986284 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:52.987661 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:52.988309 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:52.988439 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:52.989464 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:52.989470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:52.989483 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:52.989497 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:52.989547 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:52.989562 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:52.989671 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:52.989858 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 04:27:52.989904 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:27:52.990504 24709 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40849 } }
I20260807 04:27:52.991055 24709 leader_election.cc:302] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef40d76c8e1b46d0b8d7c466f56c5d66; no voters:
I20260807 04:27:52.991674 24709 leader_election.cc:288] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:52.991758 24719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:52.992623 24719 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [term 1 LEADER]: Becoming Leader. State: Replica: ef40d76c8e1b46d0b8d7c466f56c5d66, State: Running, Role: LEADER
I20260807 04:27:52.993132 24719 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40849 } }
I20260807 04:27:52.994166 24709 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:52.994861 24721 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef40d76c8e1b46d0b8d7c466f56c5d66. Latest consensus state: current_term: 1 leader_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40849 } } }
I20260807 04:27:52.995343 24721 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:52.995832 24720 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef40d76c8e1b46d0b8d7c466f56c5d66" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40849 } } }
I20260807 04:27:52.996194 24720 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:53.003664 24724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:53.011291 24724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:53.019943 24724 catalog_manager.cc:1347] Generated new cluster ID: 37f59c6d8b9a469ca2fc4d58258f5d19
I20260807 04:27:53.020043 24724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:53.052361 24724 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:53.054173 24724 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:53.066478 24724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef40d76c8e1b46d0b8d7c466f56c5d66: Generated new TSK 0
I20260807 04:27:53.067270 24724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:53.083459 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24641
W20260807 04:27:53.084825 24704 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33179
W20260807 04:27:53.085620 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24641: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:53.085762 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24641
W20260807 04:27:53.106351 24626 connection.cc:535] client connection to 127.1.244.254:40849 recv error: Network error: failed to read from TLS socket (remote: 127.1.244.254:40849): Connection reset by peer (error 104)
2026-08-07T04:27:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1992 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:53 dist-test-slave-zkpd krb5kdc[24741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:53 dist-test-slave-zkpd krb5kdc[24741](info): set up 2 sockets
Aug 07 04:27:53 dist-test-slave-zkpd krb5kdc[24741](info): commencing operation
krb5kdc: starting...
W20260807 04:27:54.278939 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:54 dist-test-slave-zkpd krb5kdc[24741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:54.561239 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46109
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:35637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:46109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:54.897385 24755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:54.897686 24755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:54.897778 24755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:54.904675 24755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:54.904778 24755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:54.904824 24755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:54.904886 24755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:54.904922 24755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:54.914237 24755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:35637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:46109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:46109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:54.915377 24755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:54.916611 24755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:54.928100 24760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:54.928032 24761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:54.928182 24763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:54.929098 24755 server_base.cc:1034] running on GCE node
I20260807 04:27:54.929865 24755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:54.931098 24755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:54.932291 24755 hybrid_clock.cc:648] HybridClock initialized: now 1786076874932239 us; error 51 us; skew 500 ppm
I20260807 04:27:54.932632 24755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:54 dist-test-slave-zkpd krb5kdc[24741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076874, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:54.935726 24755 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:54.937212 24755 webserver.cc:460] Webserver started at http://127.1.244.254:39657/ using document root <none> and password file <none>
I20260807 04:27:54.937852 24755 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:54.937989 24755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:54.938295 24755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:54.941220 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "c094b8d07a1e468bbfbd55e8979e1554"
format_stamp: "Formatted at 2026-08-07 04:27:54 on dist-test-slave-zkpd"
server_key: "88c783e34043ec0c2b3424953d354af1"
server_key_iv: "95b63d23cbb8fc85943f02ba7d7ad6dc"
server_key_version: "encryptionkey@0"
I20260807 04:27:54.942030 24755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "c094b8d07a1e468bbfbd55e8979e1554"
format_stamp: "Formatted at 2026-08-07 04:27:54 on dist-test-slave-zkpd"
server_key: "88c783e34043ec0c2b3424953d354af1"
server_key_iv: "95b63d23cbb8fc85943f02ba7d7ad6dc"
server_key_version: "encryptionkey@0"
I20260807 04:27:54.946981 24755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 04:27:54.950258 24770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:54.951481 24755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 04:27:54.951658 24755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "c094b8d07a1e468bbfbd55e8979e1554"
format_stamp: "Formatted at 2026-08-07 04:27:54 on dist-test-slave-zkpd"
server_key: "88c783e34043ec0c2b3424953d354af1"
server_key_iv: "95b63d23cbb8fc85943f02ba7d7ad6dc"
server_key_version: "encryptionkey@0"
I20260807 04:27:54.951860 24755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:54.957295 24755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:54.961349 24755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:54.961666 24755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:54.980042 24755 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:46109
I20260807 04:27:54.980074 24822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:46109 every 8 connection(s)
I20260807 04:27:54.981340 24755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:54.985054 24823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:54.988394 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24755
I20260807 04:27:54.988580 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:54.989029 2003 external_mini_cluster.cc:1428] Setting key a2eda9c96a69c626011e0ebf171f60db
I20260807 04:27:54.996609 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:54.990774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44565 (local address 127.1.244.254:46109)
0807 04:27:54.991587 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:54.991597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:54.991635 (+ 38us) server_negotiation.cc:407] Connection header received
0807 04:27:54.993278 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:54.993300 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:54.993634 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:54.993962 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:54.995055 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:54.995062 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:54.995076 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:54.995093 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:54.995149 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:54.995167 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:54.995336 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:54.995484 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 04:27:54.995681 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":365,"threads_started":1}
I20260807 04:27:54.997316 24823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554: Bootstrap starting.
I20260807 04:27:55.000174 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:55.000658 24823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:55.001592 24823 log.cc:824] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:55.004199 24823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554: No bootstrap required, opened a new log
I20260807 04:27:55.008210 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:55.005852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58687 (local address 127.1.244.254:46109)
0807 04:27:55.006009 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:55.006015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:55.006377 (+ 362us) server_negotiation.cc:407] Connection header received
0807 04:27:55.006511 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:55.006516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:55.006569 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:55.006672 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:55.007561 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:55.007567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:55.007571 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:55.007593 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:55.007643 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:55.007661 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:55.007781 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:55.007858 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 04:27:55.007944 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 04:27:55.012261 24823 raft_consensus.cc:348] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c094b8d07a1e468bbfbd55e8979e1554" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46109 } }
I20260807 04:27:55.012566 24823 raft_consensus.cc:374] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:55.012648 24823 raft_consensus.cc:729] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c094b8d07a1e468bbfbd55e8979e1554, State: Initialized, Role: FOLLOWER
I20260807 04:27:55.013144 24823 consensus_queue.cc:260] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c094b8d07a1e468bbfbd55e8979e1554" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46109 } }
I20260807 04:27:55.013352 24823 raft_consensus.cc:388] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:55.013445 24823 raft_consensus.cc:482] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:55.013559 24823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:55.015889 24823 raft_consensus.cc:504] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c094b8d07a1e468bbfbd55e8979e1554" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46109 } }
I20260807 04:27:55.016381 24823 leader_election.cc:302] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c094b8d07a1e468bbfbd55e8979e1554; no voters:
I20260807 04:27:55.017004 24823 leader_election.cc:288] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:55.017064 24833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:55.018965 24823 sys_catalog.cc:564] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:55.020289 24833 raft_consensus.cc:686] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [term 1 LEADER]: Becoming Leader. State: Replica: c094b8d07a1e468bbfbd55e8979e1554, State: Running, Role: LEADER
I20260807 04:27:55.020920 24833 consensus_queue.cc:237] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c094b8d07a1e468bbfbd55e8979e1554" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46109 } }
I20260807 04:27:55.023839 24834 sys_catalog.cc:455] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c094b8d07a1e468bbfbd55e8979e1554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c094b8d07a1e468bbfbd55e8979e1554" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46109 } } }
I20260807 04:27:55.024260 24834 sys_catalog.cc:458] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:55.024149 24835 sys_catalog.cc:455] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c094b8d07a1e468bbfbd55e8979e1554. Latest consensus state: current_term: 1 leader_uuid: "c094b8d07a1e468bbfbd55e8979e1554" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c094b8d07a1e468bbfbd55e8979e1554" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 46109 } } }
I20260807 04:27:55.024559 24835 sys_catalog.cc:458] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:55.027233 24845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:55.031253 24845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:55.040206 24845 catalog_manager.cc:1347] Generated new cluster ID: 79ebe6a837be483bbb401c76cd1ba5aa
I20260807 04:27:55.040484 24845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:55.052911 24845 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:55.053359 24845 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:55.070344 24845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c094b8d07a1e468bbfbd55e8979e1554: Generated new TSK 0
I20260807 04:27:55.071296 24845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:55.083737 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24755
W20260807 04:27:55.085014 24818 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44565
W20260807 04:27:55.085852 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24755: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:55.086010 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24755
2026-08-07T04:27:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2000 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:55 dist-test-slave-zkpd krb5kdc[24854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:55 dist-test-slave-zkpd krb5kdc[24854](info): set up 2 sockets
Aug 07 04:27:55 dist-test-slave-zkpd krb5kdc[24854](info): commencing operation
krb5kdc: starting...
W20260807 04:27:56.222872 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:56 dist-test-slave-zkpd krb5kdc[24854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:56.515410 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34277
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:34277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:56.846933 24869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:56.847244 24869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:56.847400 24869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:56.854494 24869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:56.854640 24869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:56.854738 24869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:56.854848 24869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:56.854923 24869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:56.864241 24869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:34277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:34277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:56.865439 24869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:56.866694 24869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:56.878697 24875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:56.878775 24874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:56.878739 24877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:56.879557 24869 server_base.cc:1034] running on GCE node
I20260807 04:27:56.880337 24869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:56.881585 24869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:56.882790 24869 hybrid_clock.cc:648] HybridClock initialized: now 1786076876882745 us; error 39 us; skew 500 ppm
I20260807 04:27:56.883131 24869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:56 dist-test-slave-zkpd krb5kdc[24854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:56.890141 24869 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:56.891549 24869 webserver.cc:460] Webserver started at http://127.1.244.254:32769/ using document root <none> and password file <none>
I20260807 04:27:56.892263 24869 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:56.892416 24869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:56.892767 24869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:56.895658 24869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "0a705c672c3b4876af30758f37e99802"
format_stamp: "Formatted at 2026-08-07 04:27:56 on dist-test-slave-zkpd"
server_key: "90c01c32cce1e95bdae71626e63ef672"
server_key_iv: "95df9b1a40b8d534e180cd35ea4684d3"
server_key_version: "encryptionkey@0"
I20260807 04:27:56.896505 24869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "0a705c672c3b4876af30758f37e99802"
format_stamp: "Formatted at 2026-08-07 04:27:56 on dist-test-slave-zkpd"
server_key: "90c01c32cce1e95bdae71626e63ef672"
server_key_iv: "95df9b1a40b8d534e180cd35ea4684d3"
server_key_version: "encryptionkey@0"
I20260807 04:27:56.901392 24869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 04:27:56.904842 24884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:56.906057 24869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:27:56.906248 24869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "0a705c672c3b4876af30758f37e99802"
format_stamp: "Formatted at 2026-08-07 04:27:56 on dist-test-slave-zkpd"
server_key: "90c01c32cce1e95bdae71626e63ef672"
server_key_iv: "95df9b1a40b8d534e180cd35ea4684d3"
server_key_version: "encryptionkey@0"
I20260807 04:27:56.906455 24869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:56.917410 24869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:56.921031 24869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:56.921346 24869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:56.939821 24869 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:34277
I20260807 04:27:56.939860 24937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:34277 every 8 connection(s)
I20260807 04:27:56.941147 24869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:56.943684 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24869
I20260807 04:27:56.943876 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:56.944381 2003 external_mini_cluster.cc:1428] Setting key baea3618e6cbc371f0cd3c0ccc14dc58
I20260807 04:27:56.944870 24938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 04:27:56 dist-test-slave-zkpd krb5kdc[24854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:27:56.959412 24938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802: Bootstrap starting.
I20260807 04:27:56.963559 24938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:56.964900 24938 log.cc:824] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:56.967716 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:56.946030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47385 (local address 127.1.244.254:34277)
0807 04:27:56.946797 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:56.946808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:56.946847 (+ 39us) server_negotiation.cc:407] Connection header received
0807 04:27:56.948364 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:56.948384 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:56.948736 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:56.949063 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:56.950853 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:56.952228 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:56.954021 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:56.954643 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:56.954763 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:56.959371 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:56.959391 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:56.959409 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:56.959453 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:56.963256 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:56.964088 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:56.964098 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:56.964111 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:56.964303 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:56.964648 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:56.964655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:56.964659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:56.965346 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:56.965703 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:56.966097 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 04:27:56.966458 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":384,"threads_started":1}
I20260807 04:27:56.968384 24938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802: No bootstrap required, opened a new log
I20260807 04:27:56.971750 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:56.977339 24938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a705c672c3b4876af30758f37e99802" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34277 } }
I20260807 04:27:56.977713 24938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:56.977836 24938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a705c672c3b4876af30758f37e99802, State: Initialized, Role: FOLLOWER
I20260807 04:27:56.978442 24938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a705c672c3b4876af30758f37e99802" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34277 } }
I20260807 04:27:56.978689 24938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:56.978847 24938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:56.979048 24938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:56.981364 24938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a705c672c3b4876af30758f37e99802" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34277 } }
I20260807 04:27:56.981878 24938 leader_election.cc:302] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a705c672c3b4876af30758f37e99802; no voters:
I20260807 04:27:56.982542 24938 leader_election.cc:288] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:56.982616 24948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:56.984895 24938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:56.988409 24948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [term 1 LEADER]: Becoming Leader. State: Replica: 0a705c672c3b4876af30758f37e99802, State: Running, Role: LEADER
I20260807 04:27:56.989054 24948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a705c672c3b4876af30758f37e99802" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34277 } }
I20260807 04:27:56.991838 24949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a705c672c3b4876af30758f37e99802. Latest consensus state: current_term: 1 leader_uuid: "0a705c672c3b4876af30758f37e99802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a705c672c3b4876af30758f37e99802" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34277 } } }
I20260807 04:27:56.992272 24949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:56.992770 24950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a705c672c3b4876af30758f37e99802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a705c672c3b4876af30758f37e99802" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 34277 } } }
I20260807 04:27:56.993052 24950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:56.993480 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:56.976090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51179 (local address 127.1.244.254:34277)
0807 04:27:56.976248 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:56.976252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:56.976714 (+ 462us) server_negotiation.cc:407] Connection header received
0807 04:27:56.976898 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:56.976904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:56.976975 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:56.977074 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:56.978608 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:56.979534 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:56.980850 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:56.981282 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:56.981420 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:56.984460 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:56.984470 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:56.984478 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:27:56.984526 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:56.989259 (+ 4733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:56.990079 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:56.990085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:56.990091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:56.990318 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:27:56.990851 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:27:56.990860 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:27:56.990864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:27:56.991264 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:56.991595 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:56.992803 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0807 04:27:56.993056 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 04:27:56.993711 24956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:57.000697 24956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:57.011193 24956 catalog_manager.cc:1347] Generated new cluster ID: cf4f7717e1b24369a265e6f188b007b5
I20260807 04:27:57.011296 24956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:57.018894 24956 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:57.020367 24956 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:57.027458 24956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a705c672c3b4876af30758f37e99802: Generated new TSK 0
I20260807 04:27:57.028265 24956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:57.038077 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24869
I20260807 04:27:57.039495 24932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:27:57.129837 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24869
2026-08-07T04:27:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2042 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:57 dist-test-slave-zkpd krb5kdc[24970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:57 dist-test-slave-zkpd krb5kdc[24970](info): set up 2 sockets
Aug 07 04:27:57 dist-test-slave-zkpd krb5kdc[24970](info): commencing operation
krb5kdc: starting...
W20260807 04:27:58.287194 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:27:58 dist-test-slave-zkpd krb5kdc[24970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:27:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:27:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:27:58.591115 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:32873
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:37019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:32873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:27:58.929919 24983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:27:58.930243 24983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:27:58.930414 24983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:27:58.937402 24983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:27:58.937567 24983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:27:58.937671 24983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:27:58.937800 24983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:27:58.937883 24983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:27:58.947381 24983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:37019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:32873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:32873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:27:58.948671 24983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:27:58.949992 24983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:27:58.961686 24991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:58.961777 24989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:27:58.962003 24988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:27:58.962772 24983 server_base.cc:1034] running on GCE node
I20260807 04:27:58.963450 24983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:27:58.964782 24983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:27:58.965986 24983 hybrid_clock.cc:648] HybridClock initialized: now 1786076878965931 us; error 50 us; skew 500 ppm
I20260807 04:27:58.966266 24983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:27:58 dist-test-slave-zkpd krb5kdc[24970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:27:58.969437 24983 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:27:58.970736 24983 webserver.cc:460] Webserver started at http://127.1.244.254:39605/ using document root <none> and password file <none>
I20260807 04:27:58.971328 24983 fs_manager.cc:359] Metadata directory not provided
I20260807 04:27:58.971426 24983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:27:58.971666 24983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:27:58.974808 24983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "0793d67b39bc4264b46c7f6b91eb6052"
format_stamp: "Formatted at 2026-08-07 04:27:58 on dist-test-slave-zkpd"
server_key: "49b21b2ec5bab3d96439d9d7a3912711"
server_key_iv: "e4ae12b60ac212f6466046c9b6572377"
server_key_version: "encryptionkey@0"
I20260807 04:27:58.975811 24983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "0793d67b39bc4264b46c7f6b91eb6052"
format_stamp: "Formatted at 2026-08-07 04:27:58 on dist-test-slave-zkpd"
server_key: "49b21b2ec5bab3d96439d9d7a3912711"
server_key_iv: "e4ae12b60ac212f6466046c9b6572377"
server_key_version: "encryptionkey@0"
I20260807 04:27:58.980964 24983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 04:27:58.984369 24998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:27:58.985582 24983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 04:27:58.985762 24983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "0793d67b39bc4264b46c7f6b91eb6052"
format_stamp: "Formatted at 2026-08-07 04:27:58 on dist-test-slave-zkpd"
server_key: "49b21b2ec5bab3d96439d9d7a3912711"
server_key_iv: "e4ae12b60ac212f6466046c9b6572377"
server_key_version: "encryptionkey@0"
I20260807 04:27:58.985961 24983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:27:58.999761 24983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:27:59.003425 24983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:27:59.003697 24983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:27:59.022351 24983 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:32873
I20260807 04:27:59.022380 25051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:32873 every 8 connection(s)
I20260807 04:27:59.023696 24983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:27:59.027415 25052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:27:59.028918 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 24983
I20260807 04:27:59.029111 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:27:59.029568 2003 external_mini_cluster.cc:1428] Setting key 63983104ef9099f34e13f3fd89bb0d3b
I20260807 04:27:59.041093 25052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052: Bootstrap starting.
I20260807 04:27:59.042665 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:59.031254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60439 (local address 127.1.244.254:32873)
0807 04:27:59.032090 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:59.032101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:59.032141 (+ 40us) server_negotiation.cc:407] Connection header received
0807 04:27:59.033731 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:59.033751 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:59.034152 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:59.034473 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:59.035092 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:59.036369 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:59.038265 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:59.038925 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:59.039054 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:59.040405 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:59.040413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:59.040426 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:59.040978 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:59.041058 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:59.041086 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:59.041185 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:59.041432 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 04:27:59.041721 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":376,"threads_started":1}
I20260807 04:27:59.044339 25052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052: Neither blocks nor log segments found. Creating new log.
I20260807 04:27:59.045492 25052 log.cc:824] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052: Log is configured to *not* fsync() on all Append() calls
I20260807 04:27:59.046398 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:27:59.048323 25052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052: No bootstrap required, opened a new log
I20260807 04:27:59.057307 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:27:59.050824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34617 (local address 127.1.244.254:32873)
0807 04:27:59.050994 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:27:59.051000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:27:59.051525 (+ 525us) server_negotiation.cc:407] Connection header received
0807 04:27:59.051666 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:27:59.051671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:27:59.051744 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:27:59.051850 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:27:59.052350 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:59.053479 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:59.054729 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:27:59.055488 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:27:59.055622 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:27:59.056599 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:27:59.056605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:27:59.056609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:27:59.056627 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:27:59.056669 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:27:59.056688 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:27:59.056779 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 04:27:59.056913 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 04:27:59.056998 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 04:27:59.057243 25052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0793d67b39bc4264b46c7f6b91eb6052" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32873 } }
I20260807 04:27:59.057687 25052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:27:59.057842 25052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0793d67b39bc4264b46c7f6b91eb6052, State: Initialized, Role: FOLLOWER
I20260807 04:27:59.058490 25052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0793d67b39bc4264b46c7f6b91eb6052" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32873 } }
I20260807 04:27:59.058763 25052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:27:59.058907 25052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:27:59.059079 25052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:27:59.061311 25052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0793d67b39bc4264b46c7f6b91eb6052" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32873 } }
I20260807 04:27:59.061780 25052 leader_election.cc:302] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0793d67b39bc4264b46c7f6b91eb6052; no voters:
I20260807 04:27:59.062472 25052 leader_election.cc:288] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:27:59.062584 25063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:27:59.063333 25063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [term 1 LEADER]: Becoming Leader. State: Replica: 0793d67b39bc4264b46c7f6b91eb6052, State: Running, Role: LEADER
I20260807 04:27:59.063944 25063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0793d67b39bc4264b46c7f6b91eb6052" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32873 } }
I20260807 04:27:59.064779 25052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:27:59.066886 25064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0793d67b39bc4264b46c7f6b91eb6052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0793d67b39bc4264b46c7f6b91eb6052" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32873 } } }
I20260807 04:27:59.067318 25064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:59.068156 25065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0793d67b39bc4264b46c7f6b91eb6052. Latest consensus state: current_term: 1 leader_uuid: "0793d67b39bc4264b46c7f6b91eb6052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0793d67b39bc4264b46c7f6b91eb6052" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 32873 } } }
I20260807 04:27:59.068609 25065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052 [sys.catalog]: This master's current role is: LEADER
I20260807 04:27:59.069176 25071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:27:59.073704 25071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:27:59.081916 25071 catalog_manager.cc:1347] Generated new cluster ID: a783b6fa39c24cfe9940ca506964e8dd
I20260807 04:27:59.082023 25071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:27:59.101989 25071 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:27:59.103875 25071 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:27:59.113899 25071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0793d67b39bc4264b46c7f6b91eb6052: Generated new TSK 0
I20260807 04:27:59.114837 25071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:27:59.131016 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24983
W20260807 04:27:59.132216 25047 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60439
W20260807 04:27:59.132884 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 24983: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:27:59.133057 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 24983
2026-08-07T04:27:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:27:59 dist-test-slave-zkpd krb5kdc[25085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:27:59 dist-test-slave-zkpd krb5kdc[25085](info): set up 2 sockets
Aug 07 04:27:59 dist-test-slave-zkpd krb5kdc[25085](info): commencing operation
krb5kdc: starting...
W20260807 04:28:00.272137 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.011s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:28:00 dist-test-slave-zkpd krb5kdc[25085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:28:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:28:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:28:00.573426 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40423
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:39655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:28:00.915724 25098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:00.916076 25098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:28:00.916246 25098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:28:00.923326 25098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:00.923446 25098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:00.923493 25098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:00.923606 25098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:00.923687 25098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:00.934042 25098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:39655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:00.935271 25098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:00.936626 25098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:28:00.948416 25106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:00.948519 25103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:00.948518 25104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:00.949723 25098 server_base.cc:1034] running on GCE node
I20260807 04:28:00.950452 25098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:00.951793 25098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:00.953027 25098 hybrid_clock.cc:648] HybridClock initialized: now 1786076880953008 us; error 51 us; skew 500 ppm
I20260807 04:28:00.953367 25098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:00 dist-test-slave-zkpd krb5kdc[25085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076880, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:00.956423 25098 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:28:00.957777 25098 webserver.cc:460] Webserver started at http://127.1.244.254:46187/ using document root <none> and password file <none>
I20260807 04:28:00.958420 25098 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:00.958554 25098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:00.958869 25098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:00.961794 25098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "b25d607f00ad4d7fab7d71a831daad0a"
format_stamp: "Formatted at 2026-08-07 04:28:00 on dist-test-slave-zkpd"
server_key: "0236341e8318d475b4b0e8ba1ce98cb3"
server_key_iv: "cb182fd3de8ab683c94f1904a9e436ce"
server_key_version: "encryptionkey@0"
I20260807 04:28:00.962625 25098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "b25d607f00ad4d7fab7d71a831daad0a"
format_stamp: "Formatted at 2026-08-07 04:28:00 on dist-test-slave-zkpd"
server_key: "0236341e8318d475b4b0e8ba1ce98cb3"
server_key_iv: "cb182fd3de8ab683c94f1904a9e436ce"
server_key_version: "encryptionkey@0"
I20260807 04:28:00.967556 25098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:28:00.970834 25114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:00.972170 25098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 04:28:00.972345 25098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "b25d607f00ad4d7fab7d71a831daad0a"
format_stamp: "Formatted at 2026-08-07 04:28:00 on dist-test-slave-zkpd"
server_key: "0236341e8318d475b4b0e8ba1ce98cb3"
server_key_iv: "cb182fd3de8ab683c94f1904a9e436ce"
server_key_version: "encryptionkey@0"
I20260807 04:28:00.972532 25098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:00.977950 25098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:00.982039 25098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:00.982342 25098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:01.000670 25098 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40423
I20260807 04:28:01.000734 25168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40423 every 8 connection(s)
I20260807 04:28:01.001997 25098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:28:01.003423 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25098
I20260807 04:28:01.003605 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:28:01.004216 2003 external_mini_cluster.cc:1428] Setting key 281c1e34a932fe5f9e9ac29036c3a699
I20260807 04:28:01.006604 25169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:01.013001 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:01.006000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43285 (local address 127.1.244.254:40423)
0807 04:28:01.006780 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:01.006795 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:01.006844 (+ 49us) server_negotiation.cc:407] Connection header received
0807 04:28:01.009196 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:01.009222 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:01.009645 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:01.010057 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:01.011057 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:01.011065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:01.011081 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:28:01.011098 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:01.011194 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:28:01.011220 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:01.011444 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:01.011579 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 04:28:01.011766 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":310,"threads_started":1}
I20260807 04:28:01.016439 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:28:01.020314 25169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a: Bootstrap starting.
I20260807 04:28:01.024237 25169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a: Neither blocks nor log segments found. Creating new log.
I20260807 04:28:01.024312 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:01.021580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47065 (local address 127.1.244.254:40423)
0807 04:28:01.021780 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:01.021788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:01.022360 (+ 572us) server_negotiation.cc:407] Connection header received
0807 04:28:01.022542 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:01.022546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:01.022588 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:01.022661 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:01.023599 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:01.023603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:01.023606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 04:28:01.023619 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:01.023653 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 04:28:01.023667 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:01.023760 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:01.023957 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 04:28:01.024000 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:28:01.025555 25169 log.cc:824] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:01.028060 25169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a: No bootstrap required, opened a new log
I20260807 04:28:01.035132 25169 raft_consensus.cc:348] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25d607f00ad4d7fab7d71a831daad0a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40423 } }
I20260807 04:28:01.035615 25169 raft_consensus.cc:374] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 04:28:01.035765 25169 raft_consensus.cc:729] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b25d607f00ad4d7fab7d71a831daad0a, State: Initialized, Role: FOLLOWER
I20260807 04:28:01.036361 25169 consensus_queue.cc:260] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25d607f00ad4d7fab7d71a831daad0a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40423 } }
I20260807 04:28:01.036592 25169 raft_consensus.cc:388] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 04:28:01.036743 25169 raft_consensus.cc:482] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 04:28:01.036924 25169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:01.039000 25169 raft_consensus.cc:504] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25d607f00ad4d7fab7d71a831daad0a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40423 } }
I20260807 04:28:01.039532 25169 leader_election.cc:302] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b25d607f00ad4d7fab7d71a831daad0a; no voters:
I20260807 04:28:01.040194 25169 leader_election.cc:288] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 04:28:01.040321 25179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:28:01.041081 25179 raft_consensus.cc:686] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [term 1 LEADER]: Becoming Leader. State: Replica: b25d607f00ad4d7fab7d71a831daad0a, State: Running, Role: LEADER
I20260807 04:28:01.041709 25179 consensus_queue.cc:237] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25d607f00ad4d7fab7d71a831daad0a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40423 } }
I20260807 04:28:01.042518 25169 sys_catalog.cc:564] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:28:01.044555 25181 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b25d607f00ad4d7fab7d71a831daad0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25d607f00ad4d7fab7d71a831daad0a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40423 } } }
I20260807 04:28:01.045012 25181 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:01.044881 25182 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b25d607f00ad4d7fab7d71a831daad0a. Latest consensus state: current_term: 1 leader_uuid: "b25d607f00ad4d7fab7d71a831daad0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b25d607f00ad4d7fab7d71a831daad0a" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40423 } } }
I20260807 04:28:01.045137 25182 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:01.046634 25187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:28:01.051738 25187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:28:01.061043 25187 catalog_manager.cc:1347] Generated new cluster ID: 93174dacb41f40b3855d6dbbada0b077
I20260807 04:28:01.061146 25187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:28:01.094558 25187 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:28:01.095000 25187 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:28:01.106850 25187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b25d607f00ad4d7fab7d71a831daad0a: Generated new TSK 0
I20260807 04:28:01.107851 25187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:28:01.117841 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25098
W20260807 04:28:01.118999 25164 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43285
W20260807 04:28:01.119752 2003 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25098: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 04:28:01.119927 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25098
2026-08-07T04:28:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1986 ms)
[----------] 18 tests from AuthTokenIssuingTest (36037 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:28:01 dist-test-slave-zkpd krb5kdc[25201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:28:01 dist-test-slave-zkpd krb5kdc[25201](info): set up 2 sockets
Aug 07 04:28:01 dist-test-slave-zkpd krb5kdc[25201](info): commencing operation
krb5kdc: starting...
W20260807 04:28:02.255360 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:28:02 dist-test-slave-zkpd krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:28:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:28:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:28:02.556095 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38213
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38213,127.1.244.253:41551,127.1.244.252:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:28:02.890547 25215 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:28:02.890869 25215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:02.891000 25215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:28:02.891135 25215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:28:02.898144 25215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:02.898221 25215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:02.898289 25215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:02.898368 25215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:02.898458 25215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:02.908083 25215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38213,127.1.244.253:41551,127.1.244.252:45739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:38213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:02.909307 25215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:02.910585 25215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:28:02.922029 25221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:02.922106 25222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:02.922067 25224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:02.922910 25215 server_base.cc:1034] running on GCE node
I20260807 04:28:02.923635 25215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:02.925017 25215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:02.926229 25215 hybrid_clock.cc:648] HybridClock initialized: now 1786076882926174 us; error 27 us; skew 500 ppm
I20260807 04:28:02.926549 25215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:02 dist-test-slave-zkpd krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:02.934093 25215 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:28:02.935561 25215 webserver.cc:460] Webserver started at http://127.1.244.254:40757/ using document root <none> and password file <none>
I20260807 04:28:02.936316 25215 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:02.936452 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:02.936775 25215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:02.939640 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "d3c2fe3e2c544a379d4c8d75e22a2782"
format_stamp: "Formatted at 2026-08-07 04:28:02 on dist-test-slave-zkpd"
server_key: "477b86b0ec4f9661889453269fc364bf"
server_key_iv: "fa39f4b367640ee82c4ff21fd3d25b6b"
server_key_version: "encryptionkey@0"
I20260807 04:28:02.940475 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "d3c2fe3e2c544a379d4c8d75e22a2782"
format_stamp: "Formatted at 2026-08-07 04:28:02 on dist-test-slave-zkpd"
server_key: "477b86b0ec4f9661889453269fc364bf"
server_key_iv: "fa39f4b367640ee82c4ff21fd3d25b6b"
server_key_version: "encryptionkey@0"
I20260807 04:28:02.945331 25215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 04:28:02.948673 25231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:02.949900 25215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:28:02.950080 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "d3c2fe3e2c544a379d4c8d75e22a2782"
format_stamp: "Formatted at 2026-08-07 04:28:02 on dist-test-slave-zkpd"
server_key: "477b86b0ec4f9661889453269fc364bf"
server_key_iv: "fa39f4b367640ee82c4ff21fd3d25b6b"
server_key_version: "encryptionkey@0"
I20260807 04:28:02.950292 25215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:02.970440 25215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:02.974187 25215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:02.974462 25215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:02.993024 25215 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:38213
I20260807 04:28:02.993065 25283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:38213 every 8 connection(s)
I20260807 04:28:02.994328 25215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:28:02.997290 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25215
I20260807 04:28:02.997531 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:28:02.998310 25284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:02.998384 2003 external_mini_cluster.cc:1428] Setting key 6d51ac9ac665bc4ba2be790cb5e94e95
I20260807 04:28:03.008260 25284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:03 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.253@KRBTEST.COM" created.
I20260807 04:28:03.032413 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.012183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38213 (local address 127.0.0.1:37940)
0807 04:28:03.014629 (+ 2446us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.014670 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:03.016442 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:03.017245 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:03.017251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:03.017970 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:03.018296 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.018312 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.019928 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.019934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.020962 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.020973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.021899 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.021915 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.022123 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.022781 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:03.022808 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:03.025002 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:03.028314 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.028327 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.028337 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.028719 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.029128 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.029133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.029137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.029286 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.030014 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:03.030022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:03.030231 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 04:28:03.030523 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 04:28:03.031063 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2236,"mutex_wait_us":141,"spinlock_wait_cycles":62336,"thread_start_us":2023,"threads_started":1}
I20260807 04:28:03.032439 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.012183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37940 (local address 127.1.244.254:38213)
0807 04:28:03.013067 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:03.013078 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:03.014800 (+ 1722us) server_negotiation.cc:407] Connection header received
0807 04:28:03.016806 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:03.016812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:03.017019 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:03.017228 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:03.018521 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.019749 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.021101 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.021633 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.021868 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.025218 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:03.025226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:03.025236 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:03.025267 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:03.028181 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.028870 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.028876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.028878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.029019 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.029398 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.029402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.029415 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.029848 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:03.030075 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:03.030647 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 04:28:03.030944 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":449,"threads_started":1}
I20260807 04:28:03.035502 25284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:03.036693 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.036293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53424)
0807 04:28:03.036462 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.036584 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 04:28:03.036957 25235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
I20260807 04:28:03.038529 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.038091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53434)
0807 04:28:03.038259 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.038362 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:28:03.038733 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:41551: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
I20260807 04:28:03.067116 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:41551
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38213,127.1.244.253:41551,127.1.244.252:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:28:03.088037 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } attempt: 1
I20260807 04:28:03.089236 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.088799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53448)
0807 04:28:03.088989 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.089108 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 04:28:03.090184 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.089850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53452)
0807 04:28:03.089996 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.090066 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:28:03.090369 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:41551: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
I20260807 04:28:03.158612 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } attempt: 2
I20260807 04:28:03.159782 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.159282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53464)
0807 04:28:03.159545 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.159643 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 04:28:03.160756 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.160408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53468)
0807 04:28:03.160574 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.160635 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:28:03.160943 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:41551: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
I20260807 04:28:03.252172 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } attempt: 3
I20260807 04:28:03.253454 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.252853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53480)
0807 04:28:03.253109 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.253232 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 04:28:03.254650 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.254232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53488)
0807 04:28:03.254393 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.254480 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 04:28:03.254864 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:41551: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
I20260807 04:28:03.398129 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } attempt: 4
I20260807 04:28:03.399400 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.398807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53492)
0807 04:28:03.399114 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.399222 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260807 04:28:03.400520 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.400141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53494)
0807 04:28:03.400293 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.400374 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:28:03.400702 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:41551: Network error: Client connection negotiation failed: client connection to 127.1.244.253:41551: connect: Connection refused (error 111)
W20260807 04:28:03.402390 25291 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:28:03.402673 25291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:03.402752 25291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:28:03.402834 25291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:28:03.410181 25291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:03.410344 25291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:03.410468 25291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:03.410552 25291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:03.410648 25291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:03.421255 25291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38213,127.1.244.253:41551,127.1.244.252:45739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:41551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:03.422621 25291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:03.423987 25291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:28:03.436208 25296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:03.436312 25299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:03.436558 25297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:03.437776 25291 server_base.cc:1034] running on GCE node
I20260807 04:28:03.438501 25291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:03.439796 25291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:03.441040 25291 hybrid_clock.cc:648] HybridClock initialized: now 1786076883440964 us; error 50 us; skew 500 ppm
I20260807 04:28:03.441354 25291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:03 dist-test-slave-zkpd krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:03.444583 25291 init.cc:377] Logged in from keytab as kudu/127.1.244.253@KRBTEST.COM (short username kudu)
I20260807 04:28:03.446020 25291 webserver.cc:460] Webserver started at http://127.1.244.253:43541/ using document root <none> and password file <none>
I20260807 04:28:03.446668 25291 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:03.446756 25291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:03.447028 25291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:03.449962 25291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data/instance:
uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213"
format_stamp: "Formatted at 2026-08-07 04:28:03 on dist-test-slave-zkpd"
server_key: "59ea3b5a93100970276e5ddc2a6a5649"
server_key_iv: "0c023ac7a24f12c4fba83fe584948b18"
server_key_version: "encryptionkey@0"
I20260807 04:28:03.450687 25291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal/instance:
uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213"
format_stamp: "Formatted at 2026-08-07 04:28:03 on dist-test-slave-zkpd"
server_key: "59ea3b5a93100970276e5ddc2a6a5649"
server_key_iv: "0c023ac7a24f12c4fba83fe584948b18"
server_key_version: "encryptionkey@0"
I20260807 04:28:03.455832 25291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 04:28:03.459604 25306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:03.461493 25291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:28:03.461676 25291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal
uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213"
format_stamp: "Formatted at 2026-08-07 04:28:03 on dist-test-slave-zkpd"
server_key: "59ea3b5a93100970276e5ddc2a6a5649"
server_key_iv: "0c023ac7a24f12c4fba83fe584948b18"
server_key_version: "encryptionkey@0"
I20260807 04:28:03.461971 25291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:03.478788 25291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:03.482532 25291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:03.482825 25291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:03.501389 25291 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:41551
I20260807 04:28:03.501423 25358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:41551 every 8 connection(s)
I20260807 04:28:03.502691 25291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
I20260807 04:28:03.503859 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25291
I20260807 04:28:03.504081 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-1/wal/instance
I20260807 04:28:03.504604 2003 external_mini_cluster.cc:1428] Setting key 73c01170b93a235a0d4477f600407c63
I20260807 04:28:03.506650 25359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:03.512900 25359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:03 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:28:03.535329 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.515448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.1.244.254:38213)
0807 04:28:03.515682 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:03.515690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:03.516878 (+ 1188us) server_negotiation.cc:407] Connection header received
0807 04:28:03.519306 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:03.519311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:03.519380 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:03.519482 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:03.521585 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.522247 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.524768 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.525168 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.525259 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.529502 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:03.529507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:03.529510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:03.529542 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:03.532220 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.533073 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.533077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.533079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.533198 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.533640 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.533643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.533645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.533929 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:03.534089 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:03.534777 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 04:28:03.535006 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 04:28:03.536365 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.515873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38213 (local address 127.0.0.1:37950)
0807 04:28:03.516705 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.516750 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:03.519011 (+ 2261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:03.519735 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:03.519747 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:03.520547 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:03.521402 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.521431 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.522418 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.522426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.524611 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.524630 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.525425 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.525439 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.525799 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.526545 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:03.526579 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:03.529256 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:03.532352 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.532361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.532376 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.532931 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.533321 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.533326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.533330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.533511 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.534062 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:03.534069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:03.534413 (+ 344us) client_negotiation.cc:769] Sending connection context
0807 04:28:03.534673 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 04:28:03.535172 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":342,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.252@KRBTEST.COM; defaulting to no policy
I20260807 04:28:03.537945 25359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.252@KRBTEST.COM" created.
Aug 07 04:28:03 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:03.554774 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.538672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53498)
0807 04:28:03.538833 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.538854 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:03.538963 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:03.540136 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:03.540148 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:03.540662 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:03.540867 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.540882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.542235 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.542243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.543688 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.543707 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.544545 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.544553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.544793 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.545376 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:03.545402 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:03.547780 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:03.551815 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.551821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.551826 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.552394 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.552765 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.552772 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.552779 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.553003 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.553592 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:03.553600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:03.553786 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 04:28:03.553954 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 04:28:03.554277 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 04:28:03.554785 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.538947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.1.244.253:41551)
0807 04:28:03.539644 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:03.539649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:03.539674 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:28:03.539719 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:03.539725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:03.539978 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:03.540096 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:03.541051 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.542080 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.543829 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.544394 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.544490 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.548027 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:03.548043 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:03.548061 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:03.548103 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:03.551679 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.552522 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.552529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.552531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.552639 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.553109 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.553112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.553114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.553421 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:03.553639 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:03.554116 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 04:28:03.554394 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":546,"thread_start_us":395,"threads_started":1}
I20260807 04:28:03.557325 25359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:03.558662 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.558089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41040)
0807 04:28:03.558300 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.558456 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 04:28:03.558980 25308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
I20260807 04:28:03.560513 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.560217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41042)
0807 04:28:03.560341 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.560421 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:28:03.560843 25359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
I20260807 04:28:03.576786 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:45739
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:38213,127.1.244.253:41551,127.1.244.252:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:28:03.610177 25359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 1
I20260807 04:28:03.611367 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.610895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41044)
0807 04:28:03.611104 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.611207 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 04:28:03.612349 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.612034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41058)
0807 04:28:03.612189 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.612255 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 04:28:03.612534 25359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
I20260807 04:28:03.680727 25359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 2
I20260807 04:28:03.681950 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.681467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41070)
0807 04:28:03.681683 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.681782 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 04:28:03.682895 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.682571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41086)
0807 04:28:03.682724 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.682790 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:28:03.683085 25359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
I20260807 04:28:03.699957 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } attempt: 5
Aug 07 04:28:03 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:03.714491 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.700614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53506)
0807 04:28:03.700790 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.700809 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:03.700926 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:03.701335 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:03.701338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:03.701692 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:03.701890 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.701898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.703430 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.703434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.704356 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:03.704369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.705308 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.705312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:03.705461 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.706117 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:03.706135 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:03.708372 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:03.712171 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.712175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.712178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.712572 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.713020 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:03.713023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:03.713025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:03.713147 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:03.713821 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:03.713824 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:03.713956 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 04:28:03.714069 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 04:28:03.714256 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 04:28:03.714730 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.700707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.1.244.253:41551)
0807 04:28:03.700923 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:03.700931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:03.700959 (+ 28us) server_negotiation.cc:407] Connection header received
0807 04:28:03.701031 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:03.701039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:03.701167 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:03.701315 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:03.702082 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.703300 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.704512 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:03.705169 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:03.705297 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:03.708568 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:03.708575 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:03.708580 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:03.708617 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:03.712041 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.712703 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.712711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.712716 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.712905 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:03.713263 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:03.713271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:03.713277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:03.713704 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:03.713860 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:03.714208 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 04:28:03.714438 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:28:03.715612 25284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:03.716863 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.716409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41098)
0807 04:28:03.716596 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.716687 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 04:28:03.717965 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.717548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41112)
0807 04:28:03.717725 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.717805 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 04:28:03.718173 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
I20260807 04:28:03.769428 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 1
I20260807 04:28:03.770680 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.770123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41114)
0807 04:28:03.770349 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.770472 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 04:28:03.771746 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.771376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41116)
0807 04:28:03.771525 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.771620 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 04:28:03.772011 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
I20260807 04:28:03.774269 25359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 3
I20260807 04:28:03.775287 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.774864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41126)
0807 04:28:03.775074 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.775155 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 04:28:03.776384 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.775975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41142)
0807 04:28:03.776202 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.776266 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260807 04:28:03.776582 25359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
I20260807 04:28:03.840233 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 2
I20260807 04:28:03.841591 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.840983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41158)
0807 04:28:03.841248 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.841367 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 04:28:03.842590 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.842254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41160)
0807 04:28:03.842420 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.842486 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 04:28:03.842873 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
W20260807 04:28:03.919155 25367 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:28:03.919481 25367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:03.919545 25367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:28:03.919713 25367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 04:28:03.919823 25359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 4
I20260807 04:28:03.921049 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.920480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41174)
0807 04:28:03.920729 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.920837 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 04:28:03.922055 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.921692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41190)
0807 04:28:03.921839 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.921926 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:28:03.922271 25359 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
W20260807 04:28:03.927699 25367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:03.927801 25367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:03.927920 25367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:03.928014 25367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:03.928103 25367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:03.937924 25367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:38213,127.1.244.253:41551,127.1.244.252:45739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:45739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:03.939141 25367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:03.940451 25367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 04:28:03.949108 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 3
I20260807 04:28:03.950376 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.949727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41204)
0807 04:28:03.950019 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.950154 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 04:28:03.952054 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:03.951348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41218)
0807 04:28:03.951747 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:03.951881 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
W20260807 04:28:03.952204 25375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:03.952353 25284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:45739: Network error: Client connection negotiation failed: client connection to 127.1.244.252:45739: connect: Connection refused (error 111)
W20260807 04:28:03.952322 25373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:03.952448 25372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:03.953220 25367 server_base.cc:1034] running on GCE node
I20260807 04:28:03.954123 25367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:03.955506 25367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:03.956745 25367 hybrid_clock.cc:648] HybridClock initialized: now 1786076883956684 us; error 30 us; skew 500 ppm
I20260807 04:28:03.957098 25367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:03 dist-test-slave-zkpd krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:03.960233 25367 init.cc:377] Logged in from keytab as kudu/127.1.244.252@KRBTEST.COM (short username kudu)
I20260807 04:28:03.961594 25367 webserver.cc:460] Webserver started at http://127.1.244.252:39877/ using document root <none> and password file <none>
I20260807 04:28:03.962194 25367 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:03.962289 25367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:03.962538 25367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:03.965412 25367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data/instance:
uuid: "10ff190bf36b40f58d8a9fcd959d9454"
format_stamp: "Formatted at 2026-08-07 04:28:03 on dist-test-slave-zkpd"
server_key: "487611f74a799e3e5320787a691a3746"
server_key_iv: "f0b08a726d8c85ef786b069e56bf42cf"
server_key_version: "encryptionkey@0"
I20260807 04:28:03.966120 25367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal/instance:
uuid: "10ff190bf36b40f58d8a9fcd959d9454"
format_stamp: "Formatted at 2026-08-07 04:28:03 on dist-test-slave-zkpd"
server_key: "487611f74a799e3e5320787a691a3746"
server_key_iv: "f0b08a726d8c85ef786b069e56bf42cf"
server_key_version: "encryptionkey@0"
I20260807 04:28:03.971295 25367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 04:28:03.974776 25382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:03.976212 25367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:28:03.976351 25367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal
uuid: "10ff190bf36b40f58d8a9fcd959d9454"
format_stamp: "Formatted at 2026-08-07 04:28:03 on dist-test-slave-zkpd"
server_key: "487611f74a799e3e5320787a691a3746"
server_key_iv: "f0b08a726d8c85ef786b069e56bf42cf"
server_key_version: "encryptionkey@0"
I20260807 04:28:03.976531 25367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:04.007625 25367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:04.011390 25367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:04.011613 25367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:04.030212 25367 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:45739
I20260807 04:28:04.030232 25434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:45739 every 8 connection(s)
I20260807 04:28:04.031668 25367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
I20260807 04:28:04.035293 25435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:04.035516 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25367
I20260807 04:28:04.035708 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786076526463677-2003-0/minicluster-data/master-2/wal/instance
I20260807 04:28:04.036235 2003 external_mini_cluster.cc:1428] Setting key 625c3bdd6053b414790a525043301d6c
I20260807 04:28:04.044307 25435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:28:04.060370 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.038160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.1.244.252:45739)
0807 04:28:04.039009 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.039021 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.039071 (+ 50us) server_negotiation.cc:407] Connection header received
0807 04:28:04.041220 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.041241 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.041631 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.041945 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.043737 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.045041 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.046806 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.048023 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.048234 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.053113 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.053119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.053128 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.053161 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.056838 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.057493 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.057507 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.057513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.057647 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.058009 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.058014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.058017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.058445 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.058726 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.059104 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.059456 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":409,"threads_started":1}
I20260807 04:28:04.062244 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.046941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:38213 (local address 127.0.0.1:37956)
0807 04:28:04.047773 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:04.047806 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:04.048008 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:04.048444 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:04.048448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:04.049173 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:04.049422 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.049431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.050933 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.050936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.052314 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.052327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.053324 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.053333 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.053564 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.054329 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:04.054347 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:04.056681 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:04.059757 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.059767 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.059774 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.060216 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.060603 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.060608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.060612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.060772 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.061323 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:04.061332 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:04.061499 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 04:28:04.061704 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 04:28:04.061956 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":477,"threads_started":1}
I20260807 04:28:04.062841 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.046876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37956 (local address 127.1.244.254:38213)
0807 04:28:04.047440 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.047448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.047927 (+ 479us) server_negotiation.cc:407] Connection header received
0807 04:28:04.048144 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.048149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.048237 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.048354 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.049582 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.050792 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.052480 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.053133 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.053224 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.056880 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.056885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.056889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.056920 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.059639 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.060341 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.060346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.060347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.060490 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.060918 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.060921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.060923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.061187 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.061298 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.061846 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.062073 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":285,"threads_started":1}
I20260807 04:28:04.064592 25435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:04.078102 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.063563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.1.244.254:38213)
0807 04:28:04.063725 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.063728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.063743 (+ 15us) server_negotiation.cc:407] Connection header received
0807 04:28:04.063961 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.063966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.064070 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.064160 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.065562 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.066423 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.067779 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.068212 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.068305 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.072525 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.072533 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.072540 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.072575 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.075414 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.076228 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.076234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.076237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.076385 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.076743 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.076750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.076756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.077104 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.077367 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.077621 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.077773 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:28:04.078670 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.065387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:41551 (local address 127.0.0.1:53512)
0807 04:28:04.065635 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:04.065649 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:04.065727 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:04.066035 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:04.066038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:04.066427 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:04.066607 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.066619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.067625 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.067630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.068878 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.068891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.069584 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.069589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.069762 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.070286 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:04.070302 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:04.072706 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:04.076229 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.076234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.076238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.076724 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.077100 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.077105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.077109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.077271 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.077870 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:04.077875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:04.078037 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 04:28:04.078193 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 04:28:04.078431 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260807 04:28:04.079007 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.065334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.1.244.253:41551)
0807 04:28:04.065526 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.065534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.065719 (+ 185us) server_negotiation.cc:407] Connection header received
0807 04:28:04.065809 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.065821 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.065904 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.066008 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.066752 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.067461 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.069027 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.069427 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.069557 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.072891 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.072896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.072899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.072929 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.076087 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.076837 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.076842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.076844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.076971 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.077395 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.077401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.077404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.077736 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.077884 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.078418 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.078625 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:28:04.079950 25435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:04.092226 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.080651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41236)
0807 04:28:04.080890 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:04.080908 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:04.080998 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:04.081263 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:04.081267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:04.081587 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:04.081695 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.081702 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.082517 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.082524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.083258 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.083268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.083919 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.083924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.084165 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.084810 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:04.084825 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:04.086686 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:04.089828 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.089834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.089838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.090177 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.090587 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.090592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.090595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.090738 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.091208 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:04.091212 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:04.091345 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 04:28:04.091509 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 04:28:04.091802 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":1}
I20260807 04:28:04.092267 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.080689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.1.244.252:45739)
0807 04:28:04.080890 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.080895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.080951 (+ 56us) server_negotiation.cc:407] Connection header received
0807 04:28:04.081102 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.081106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.081159 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.081249 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.081816 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.082406 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.083382 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.083754 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.083922 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.086852 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.086857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.086860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.086887 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.089696 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.090287 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.090292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.090295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.090455 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.090837 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.090844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.090847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.091097 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.091240 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.091589 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.091960 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":158,"server-negotiator.queue_time_us":110}
I20260807 04:28:04.093144 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.080098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.1.244.253:41551)
0807 04:28:04.080278 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.080282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.080295 (+ 13us) server_negotiation.cc:407] Connection header received
0807 04:28:04.080484 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.080488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.080586 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.080669 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.081692 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.082321 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.083416 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.083783 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.083929 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.087972 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.087976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.087979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.088006 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.090548 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.091165 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.091172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.091175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.091314 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.091719 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.091725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.091728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.092054 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.092511 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.092629 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.092841 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 04:28:04.098048 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:28:04.098031 25435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: Bootstrap starting.
I20260807 04:28:04.101755 25435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: Neither blocks nor log segments found. Creating new log.
I20260807 04:28:04.102825 25435 log.cc:824] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:04.101433 25258 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37962
I20260807 04:28:04.105573 25435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: No bootstrap required, opened a new log
I20260807 04:28:04.112397 25435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.112674 25435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ff190bf36b40f58d8a9fcd959d9454, State: Initialized, Role: FOLLOWER
I20260807 04:28:04.113209 25435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.114110 25442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.114444 25442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.115263 25435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:28:04.124593 25453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:28:04.124735 25453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:28:04.129602 25284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 4
I20260807 04:28:04.122292 25367 master_runner.cc:186] Error getting master registration for 127.1.244.254:38213: OK, instance_id { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" instance_seqno: 1786076882987811 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:28:04.132465 25367 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076882, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260807 04:28:04.147390 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.130172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41238)
0807 04:28:04.130349 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:04.130369 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:04.130484 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:04.130789 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:04.130796 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:04.131309 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:04.131522 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.131536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.133092 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.133098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.134646 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.134664 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.135730 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.135736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.135997 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.136698 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:04.136723 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:04.139451 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:04.143556 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.143561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.143566 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.144085 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.144509 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.144514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.144518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.144682 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.145432 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:04.145437 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:04.145616 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 04:28:04.146686 (+ 1070us) client_negotiation.cc:240] Negotiation successful
0807 04:28:04.146981 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 04:28:04.147410 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.130299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.1.244.252:45739)
0807 04:28:04.130489 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.130495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.130515 (+ 20us) server_negotiation.cc:407] Connection header received
0807 04:28:04.130571 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.130575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.130642 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.130736 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.131671 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.132926 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.134794 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.135566 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.135692 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.139660 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.139667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.139672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.139725 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.143414 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.144208 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.144214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.144218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.144386 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.144797 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.144802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.144806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.145269 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.145453 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.146819 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.147051 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 04:28:04.153445 25284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782: Bootstrap starting.
I20260807 04:28:04.156744 25284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782: Neither blocks nor log segments found. Creating new log.
I20260807 04:28:04.157797 25284 log.cc:824] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:04.160589 25284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782: No bootstrap required, opened a new log
I20260807 04:28:04.167793 25284 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.168272 25284 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3c2fe3e2c544a379d4c8d75e22a2782, State: Initialized, Role: FOLLOWER
I20260807 04:28:04.168711 25284 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.169689 25456 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.170068 25456 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.170749 25284 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:28:04.170907 25258 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 04:28:04.171151 25258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:04.174670 25258 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.176566 25258 leader_election.cc:288] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [CANDIDATE]: Term 1 election: Requested vote from peers 8c1d7cc0e7b1472c96bb58d6bcd17213 (127.1.244.253:41551), 10ff190bf36b40f58d8a9fcd959d9454 (127.1.244.252:45739)
I20260807 04:28:04.178282 25410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "10ff190bf36b40f58d8a9fcd959d9454"
I20260807 04:28:04.178790 25410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:04.182240 25410 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2fe3e2c544a379d4c8d75e22a2782 in term 1.
I20260807 04:28:04.182940 25233 leader_election.cc:302] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ff190bf36b40f58d8a9fcd959d9454, d3c2fe3e2c544a379d4c8d75e22a2782; no voters:
I20260807 04:28:04.183497 25456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:28:04.190913 25456 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [term 1 LEADER]: Becoming Leader. State: Replica: d3c2fe3e2c544a379d4c8d75e22a2782, State: Running, Role: LEADER
I20260807 04:28:04.191951 25456 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.204794 25464 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3c2fe3e2c544a379d4c8d75e22a2782. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.205277 25464 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:04.219844 25477 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 04:28:04.221472 25359 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } attempt: 5
W20260807 04:28:04.227246 25237 tablet.cc:2369] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:28:04.228960 25472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:28:04.234427 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.205849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.1.244.254:38213)
0807 04:28:04.206041 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.206050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.211223 (+ 5173us) server_negotiation.cc:407] Connection header received
0807 04:28:04.211649 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.211656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.211738 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.211810 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.215963 (+ 4153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.217688 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.222002 (+ 4314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.222709 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.222823 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.226595 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.226602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.226608 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.226649 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.230828 (+ 4179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.231712 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.231718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.231721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.231915 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.232385 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.232390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.232394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.232805 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.233121 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.233743 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.233988 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:28:04.225172 25215 master_runner.cc:186] Error getting master registration for 127.1.244.253:41551: OK, instance_id { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" instance_seqno: 1786076883496315 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:28:04.239130 25215 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:28:04.243418 25472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:28:04.256839 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.208301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41244 (local address 127.1.244.252:45739)
0807 04:28:04.208534 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.208542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.217046 (+ 8504us) server_negotiation.cc:407] Connection header received
0807 04:28:04.217582 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.217591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.217699 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.217783 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.220704 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.221714 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.244369 (+ 22655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.245136 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.245275 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.248476 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.248483 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.248487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.248529 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.252355 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.253730 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.253736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.253739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.253906 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.254785 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.254790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.254793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.255109 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.255333 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.256201 (+ 868us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.256461 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 04:28:04.257977 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.211161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.1.244.253:41551)
0807 04:28:04.211359 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.211368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.221087 (+ 9719us) server_negotiation.cc:407] Connection header received
0807 04:28:04.221986 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.221993 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.222076 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.222142 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.231287 (+ 9145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.232426 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.238432 (+ 6006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.239107 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.239206 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.245970 (+ 6764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.245978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.245983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.246027 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.250049 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.252008 (+ 1959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.252015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.252018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.252203 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.252668 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.252673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.252677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.253067 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.253302 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.257358 (+ 4056us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.257604 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 04:28:04 dist-test-slave-zkpd krb5kdc[25201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076883, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
W20260807 04:28:04.263592 25388 tablet.cc:2369] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:28:04.271750 25410 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [term 1 FOLLOWER]: Refusing update from remote peer d3c2fe3e2c544a379d4c8d75e22a2782: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:28:04.271775 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.222010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:45739 (local address 127.0.0.1:41254)
0807 04:28:04.222357 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:04.222377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:04.222470 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:04.237284 (+ 14814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:04.237290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:04.237783 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:04.238004 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.238016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.254344 (+ 16328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.254351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.255770 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:04.255785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.256870 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.256877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:04.257156 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.257789 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:04.257811 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:04.260718 (+ 2907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:04.267298 (+ 6580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.267304 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.267309 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.267931 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.268782 (+ 851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:04.268787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:04.268791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:04.269046 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:04.270086 (+ 1040us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:04.270113 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:04.270336 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 04:28:04.270605 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 04:28:04.271075 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260807 04:28:04.273255 25464 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:28:04.275035 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:04.222667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.1.244.252:45739)
0807 04:28:04.236635 (+ 13968us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.236643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.236706 (+ 63us) server_negotiation.cc:407] Connection header received
0807 04:28:04.236836 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.236841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.237029 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.238229 (+ 1200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.244877 (+ 6648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.254186 (+ 9309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.256030 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:04.256693 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:04.257460 (+ 767us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:04.261031 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:04.261063 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:04.261069 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:04.261132 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:04.267086 (+ 5954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.268259 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.268265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.268271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.268579 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:04.269269 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:04.269300 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:04.269303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:04.269902 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:04.272144 (+ 2242us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:04.272489 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 04:28:04.273284 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13675,"thread_start_us":13097,"threads_started":1}
I20260807 04:28:04.287101 25359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: Bootstrap starting.
I20260807 04:28:04.291236 25359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: Neither blocks nor log segments found. Creating new log.
I20260807 04:28:04.293203 25359 log.cc:824] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:04.298271 25442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3c2fe3e2c544a379d4c8d75e22a2782. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.298612 25442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.304282 25480 mvcc.cc:205] Tried to move back new op lower bound from 7315770917966454784 to 7315770917659688960. Current Snapshot: MvccSnapshot[applied={T|T < 7315770917966454784}]
I20260807 04:28:04.305470 25359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: No bootstrap required, opened a new log
I20260807 04:28:04.310979 25481 mvcc.cc:205] Tried to move back new op lower bound from 7315770917966454784 to 7315770917659688960. Current Snapshot: MvccSnapshot[applied={T|T < 7315770917966454784}]
I20260807 04:28:04.315395 25442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.315644 25442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.319229 25472 catalog_manager.cc:1347] Generated new cluster ID: f3a04f77cc044aad9dcdc16cc1c76248
I20260807 04:28:04.319084 25464 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.319327 25472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:28:04.319368 25464 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:04.320246 25359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.320644 25359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1d7cc0e7b1472c96bb58d6bcd17213, State: Initialized, Role: FOLLOWER
I20260807 04:28:04.321137 25359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } }
I20260807 04:28:04.328138 25486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.330446 25486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.330349 25333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213"
I20260807 04:28:04.330300 25359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:28:04.330809 25333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:04.335302 25333 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3c2fe3e2c544a379d4c8d75e22a2782 in term 1.
I20260807 04:28:04.336172 25332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [term 1 FOLLOWER]: Refusing update from remote peer d3c2fe3e2c544a379d4c8d75e22a2782: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:28:04.338454 25456 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:28:04.343382 25472 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:28:04.345477 25472 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:28:04.379617 25472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782: Generated new TSK 0
I20260807 04:28:04.380825 25472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 04:28:04.386638 25486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3c2fe3e2c544a379d4c8d75e22a2782. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.386983 25486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.386662 25496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.387116 25496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:04.388206 25464 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3c2fe3e2c544a379d4c8d75e22a2782" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 38213 } } peers { permanent_uuid: "8c1d7cc0e7b1472c96bb58d6bcd17213" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 41551 } } peers { permanent_uuid: "10ff190bf36b40f58d8a9fcd959d9454" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 45739 } } }
I20260807 04:28:04.388478 25464 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3c2fe3e2c544a379d4c8d75e22a2782 [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:04.410634 25504 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260807 04:28:04.488595 25478 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.475221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.1.244.252:45739)
0807 04:28:04.475386 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.475393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.486489 (+ 11096us) server_negotiation.cc:407] Connection header received
0807 04:28:04.486925 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.486932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.487041 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.487118 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.488359 (+ 1241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41266: BlockingRecv error: recv got EOF from 127.0.0.1:41266 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 04:28:04.488620 25439 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.474826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.1.244.254:38213)
0807 04:28:04.475053 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.475059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.486602 (+ 11543us) server_negotiation.cc:407] Connection header received
0807 04:28:04.487060 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.487066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.487166 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.487239 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.488387 (+ 1148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37994: BlockingRecv error: recv got EOF from 127.0.0.1:37994 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 04:28:04.490720 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.480879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.1.244.253:41551)
0807 04:28:04.481108 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.481116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.489301 (+ 8185us) server_negotiation.cc:407] Connection header received
0807 04:28:04.489532 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.489538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.489615 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.489681 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.490516 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53540: BlockingRecv error: recv got EOF from 127.0.0.1:53540 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 04:28:04.498111 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.496124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.1.244.253:41551)
0807 04:28:04.496279 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.496284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.496758 (+ 474us) server_negotiation.cc:407] Connection header received
0807 04:28:04.496932 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.496938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.497057 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.497171 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.497885 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53542: BlockingRecv error: recv got EOF from 127.0.0.1:53542 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 04:28:04.532464 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.529685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.1.244.253:41551)
0807 04:28:04.529868 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.529873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.530849 (+ 976us) server_negotiation.cc:407] Connection header received
0807 04:28:04.531086 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.531092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.531183 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.531301 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.532250 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53550: BlockingRecv error: recv got EOF from 127.0.0.1:53550 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 04:28:04.547478 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.545227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.1.244.253:41551)
0807 04:28:04.545401 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.545406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.546021 (+ 615us) server_negotiation.cc:407] Connection header received
0807 04:28:04.546208 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.546213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.546297 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.546412 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.547230 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53558: BlockingRecv error: recv got EOF from 127.0.0.1:53558 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 04:28:04.557773 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.555614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.1.244.253:41551)
0807 04:28:04.555827 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.555833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.556391 (+ 558us) server_negotiation.cc:407] Connection header received
0807 04:28:04.556547 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.556553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.556640 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.556748 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.557550 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53564: BlockingRecv error: recv got EOF from 127.0.0.1:53564 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260807 04:28:04.580044 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.577767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.1.244.253:41551)
0807 04:28:04.577947 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.577953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.578478 (+ 525us) server_negotiation.cc:407] Connection header received
0807 04:28:04.578691 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.578698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.578788 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.578910 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.579720 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53580: BlockingRecv error: recv got EOF from 127.0.0.1:53580 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 04:28:04.601214 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.598965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53594 (local address 127.1.244.253:41551)
0807 04:28:04.599127 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.599132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.599653 (+ 521us) server_negotiation.cc:407] Connection header received
0807 04:28:04.599854 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.599862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.600012 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.600136 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.600982 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53594: BlockingRecv error: recv got EOF from 127.0.0.1:53594 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 04:28:04.640862 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.638304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.1.244.253:41551)
0807 04:28:04.638536 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.638545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.639116 (+ 571us) server_negotiation.cc:407] Connection header received
0807 04:28:04.639354 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.639364 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.639464 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.639595 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.640553 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53606: BlockingRecv error: recv got EOF from 127.0.0.1:53606 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 04:28:04.710670 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.708753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.1.244.253:41551)
0807 04:28:04.708982 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.708988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.709354 (+ 366us) server_negotiation.cc:407] Connection header received
0807 04:28:04.709530 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.709537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.709633 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.709762 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.710448 (+ 686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53616: BlockingRecv error: recv got EOF from 127.0.0.1:53616 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 04:28:04.843765 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:04.842098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.1.244.253:41551)
0807 04:28:04.842293 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:04.842297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:04.842596 (+ 299us) server_negotiation.cc:407] Connection header received
0807 04:28:04.842760 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:04.842765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:04.842847 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:04.842943 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:04.843571 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53630: BlockingRecv error: recv got EOF from 127.0.0.1:53630 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 04:28:05.105522 25364 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:05.103646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.1.244.253:41551)
0807 04:28:05.103875 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:05.103882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:05.104151 (+ 269us) server_negotiation.cc:407] Connection header received
0807 04:28:05.104330 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:05.104334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:05.104414 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:05.104504 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:05.105264 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53644: BlockingRecv error: recv got EOF from 127.0.0.1:53644 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 04:28:05.126633 25453 catalog_manager.cc:1259] Loaded cluster ID: f3a04f77cc044aad9dcdc16cc1c76248
I20260807 04:28:05.126727 25453 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: loading cluster ID for follower catalog manager: success
I20260807 04:28:05.129087 25453 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: acquiring CA information for follower catalog manager: success
I20260807 04:28:05.130879 25453 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 10ff190bf36b40f58d8a9fcd959d9454: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:28:05.416481 25504 catalog_manager.cc:1259] Loaded cluster ID: f3a04f77cc044aad9dcdc16cc1c76248
I20260807 04:28:05.416601 25504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: loading cluster ID for follower catalog manager: success
I20260807 04:28:05.420441 25504 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: acquiring CA information for follower catalog manager: success
I20260807 04:28:05.422297 25504 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c1d7cc0e7b1472c96bb58d6bcd17213: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:28:05.633575 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:05.625429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.1.244.253:41551)
0807 04:28:05.627870 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:05.627877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:05.627935 (+ 58us) server_negotiation.cc:407] Connection header received
0807 04:28:05.628012 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:05.628017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:05.628157 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:05.628301 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:05.628799 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.629655 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.631149 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.631675 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.631806 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:05.632342 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:05.632889 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:05.633087 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:05.633157 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 04:28:05.633246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2268,"thread_start_us":2129,"threads_started":1}
I20260807 04:28:05.652423 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:05.646689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.1.244.252:45739)
0807 04:28:05.647362 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:05.647370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:05.647389 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:28:05.647510 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:05.647514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:05.647650 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:05.647821 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:05.648257 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.648943 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.650270 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.650679 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.650822 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:05.651299 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:05.651784 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:05.651987 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:05.652051 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 04:28:05.652132 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":430,"threads_started":1}
I20260807 04:28:05.678457 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:05.659695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41288 (local address 127.1.244.252:45739)
0807 04:28:05.660661 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:05.660669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:05.670788 (+ 10119us) server_negotiation.cc:407] Connection header received
0807 04:28:05.671009 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:05.671018 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:05.671153 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:05.671271 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:05.672100 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.673264 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.675741 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.676425 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.676561 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:05.677440 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:05.677595 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:05.677661 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:05.677980 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 04:28:05.678068 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814}
I20260807 04:28:05.679126 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:05.659482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.1.244.253:41551)
0807 04:28:05.659671 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:05.659679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:05.672239 (+ 12560us) server_negotiation.cc:407] Connection header received
0807 04:28:05.672435 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:05.672445 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:05.672554 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:05.672654 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:05.673409 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.674605 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.676895 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.677619 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.677710 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:05.678327 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:05.678468 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:05.678533 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:05.678755 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 04:28:05.678854 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 04:28:05.680464 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:05.659364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.1.244.254:38213)
0807 04:28:05.672759 (+ 13395us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:05.672765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:05.672783 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:28:05.672894 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:05.672901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:05.673141 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:05.673323 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:05.674146 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.675217 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.677427 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:05.678041 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:05.678157 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:05.678851 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:05.679309 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:05.679417 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:05.679651 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 04:28:05.680159 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13259,"threads_started":1}
I20260807 04:28:05.686177 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25215
I20260807 04:28:05.687294 25279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:28:05.797771 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25215
W20260807 04:28:05.821534 25383 connection.cc:535] server connection from 127.0.0.1:41238 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 04:28:05.822412 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25291
I20260807 04:28:05.823530 25354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:28:05.927807 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25291
I20260807 04:28:05.949647 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25367
I20260807 04:28:05.950779 25430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:28:06.058219 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25367
2026-08-07T04:28:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4939 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 04:28:06 dist-test-slave-zkpd krb5kdc[25593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 04:28:06 dist-test-slave-zkpd krb5kdc[25593](info): set up 2 sockets
Aug 07 04:28:06 dist-test-slave-zkpd krb5kdc[25593](info): commencing operation
krb5kdc: starting...
W20260807 04:28:07.195235 2003 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/test-user.keytab.
Aug 07 04:28:07 dist-test-slave-zkpd krb5kdc[25593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T04:28:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T04:28:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.244.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
Entry for principal kudu/127.1.244.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab.
I20260807 04:28:07.478803 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40015
--webserver_interface=127.1.244.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40015,127.1.244.253:35089,127.1.244.252:43845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 04:28:07.818642 25606 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:28:07.819042 25606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:07.819173 25606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:28:07.819314 25606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:28:07.826309 25606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:07.826429 25606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:07.826542 25606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:07.826630 25606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:07.826720 25606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:07.836512 25606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40015,127.1.244.253:35089,127.1.244.252:43845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.254.keytab
--principal=kudu/127.1.244.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.254:40015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.244.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:07.837750 25606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:07.839038 25606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:28:07.850703 25611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:07.850759 25614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:07.850797 25612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:07.851980 25606 server_base.cc:1034] running on GCE node
I20260807 04:28:07.852679 25606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:07.853961 25606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:07.855183 25606 hybrid_clock.cc:648] HybridClock initialized: now 1786076887855124 us; error 36 us; skew 500 ppm
I20260807 04:28:07.855461 25606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:07 dist-test-slave-zkpd krb5kdc[25593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:07.858522 25606 init.cc:377] Logged in from keytab as kudu/127.1.244.254@KRBTEST.COM (short username kudu)
I20260807 04:28:07.859802 25606 webserver.cc:460] Webserver started at http://127.1.244.254:37877/ using document root <none> and password file <none>
I20260807 04:28:07.860474 25606 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:07.860561 25606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:07.860854 25606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:07.863669 25606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data/instance:
uuid: "80a32deba3724097baa35484dac26305"
format_stamp: "Formatted at 2026-08-07 04:28:07 on dist-test-slave-zkpd"
server_key: "cf3d62b92485ccaf78ee08d8eba6d278"
server_key_iv: "ebb48a40a08c969d050cdbe96df90f29"
server_key_version: "encryptionkey@0"
I20260807 04:28:07.864475 25606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal/instance:
uuid: "80a32deba3724097baa35484dac26305"
format_stamp: "Formatted at 2026-08-07 04:28:07 on dist-test-slave-zkpd"
server_key: "cf3d62b92485ccaf78ee08d8eba6d278"
server_key_iv: "ebb48a40a08c969d050cdbe96df90f29"
server_key_version: "encryptionkey@0"
I20260807 04:28:07.869319 25606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 04:28:07.872494 25621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:07.873715 25606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 04:28:07.873909 25606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal
uuid: "80a32deba3724097baa35484dac26305"
format_stamp: "Formatted at 2026-08-07 04:28:07 on dist-test-slave-zkpd"
server_key: "cf3d62b92485ccaf78ee08d8eba6d278"
server_key_iv: "ebb48a40a08c969d050cdbe96df90f29"
server_key_version: "encryptionkey@0"
I20260807 04:28:07.874110 25606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:07.903533 25606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:07.907579 25606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:07.907922 25606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:07.926373 25606 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.254:40015
I20260807 04:28:07.926393 25674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.254:40015 every 8 connection(s)
I20260807 04:28:07.927707 25606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/data/info.pb
I20260807 04:28:07.929111 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25606
I20260807 04:28:07.929291 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-0/wal/instance
I20260807 04:28:07.929740 2003 external_mini_cluster.cc:1428] Setting key e51748930eafe68552c422f2c18cf852
I20260807 04:28:07.931531 25675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:07.937976 25675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:07 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.244.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.253@KRBTEST.COM" created.
I20260807 04:28:07.967536 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:07.941066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.1.244.254:40015)
0807 04:28:07.941906 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:07.941922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:07.942460 (+ 538us) server_negotiation.cc:407] Connection header received
0807 04:28:07.945107 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:07.945113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:07.945341 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:07.945492 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:07.947148 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:07.948404 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:07.950818 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:07.951372 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:07.951499 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:07.960448 (+ 8949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:07.960457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:07.960468 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:07.960495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:07.963345 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:07.964213 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:07.964219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:07.964221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:07.964372 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:07.964833 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:07.964837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:07.964838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:07.965309 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:07.965462 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:07.966143 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 04:28:07.966497 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":596,"thread_start_us":409,"threads_started":1}
I20260807 04:28:07.967563 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:07.941065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40015 (local address 127.0.0.1:34814)
0807 04:28:07.942260 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:07.942310 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:07.944717 (+ 2407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:07.945584 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:07.945594 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:07.946441 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:07.946935 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:07.946963 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:07.948562 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:07.948570 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:07.950569 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:07.950589 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:07.951640 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:07.951650 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:07.951926 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:07.957301 (+ 5375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:07.957335 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:07.960206 (+ 2871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:07.963461 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:07.963472 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:07.963481 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:07.964081 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:07.964493 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:07.964500 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:07.964506 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:07.964716 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:07.965454 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:07.965465 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:07.965734 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 04:28:07.966022 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 04:28:07.966701 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":114,"thread_start_us":769,"threads_started":1}
I20260807 04:28:07.970569 25675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:07.971598 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:07.971239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37888)
0807 04:28:07.971391 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:07.971498 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 04:28:07.971876 25623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
I20260807 04:28:07.973366 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:07.973048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37898)
0807 04:28:07.973176 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:07.973242 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:28:07.973641 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35089: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
Entry for principal kudu/127.1.244.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab.
I20260807 04:28:07.999099 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:35089
--webserver_interface=127.1.244.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40015,127.1.244.253:35089,127.1.244.252:43845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:28:08.022977 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } attempt: 1
I20260807 04:28:08.024362 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.023693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37906)
0807 04:28:08.023984 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.024155 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 04:28:08.025552 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.025083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37914)
0807 04:28:08.025262 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.025375 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:28:08.025825 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35089: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
I20260807 04:28:08.094118 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } attempt: 2
I20260807 04:28:08.095465 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.094815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37922)
0807 04:28:08.095117 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.095257 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:28:08.096643 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.096182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37934)
0807 04:28:08.096365 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.096452 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 04:28:08.096851 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35089: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
I20260807 04:28:08.188234 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } attempt: 3
I20260807 04:28:08.189432 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.188912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37950)
0807 04:28:08.189151 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.189268 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 04:28:08.190469 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.190077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37954)
0807 04:28:08.190226 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.190312 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 04:28:08.190655 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35089: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
I20260807 04:28:08.333909 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } attempt: 4
I20260807 04:28:08.335263 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.334627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37970)
0807 04:28:08.334973 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.335080 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260807 04:28:08.335736 25682 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:28:08.336180 25682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:08.336303 25682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260807 04:28:08.336412 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.335985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37972)
0807 04:28:08.336141 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.336249 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 04:28:08.336441 25682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:28:08.336663 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.253:35089: Network error: Client connection negotiation failed: client connection to 127.1.244.253:35089: connect: Connection refused (error 111)
W20260807 04:28:08.343549 25682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:08.343659 25682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:08.343793 25682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:08.343873 25682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:08.343976 25682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:08.354182 25682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40015,127.1.244.253:35089,127.1.244.252:43845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.253.keytab
--principal=kudu/127.1.244.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.253:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.244.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:08.355422 25682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:08.356777 25682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:28:08.368171 25687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:08.368144 25688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:08.368204 25690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:08.368973 25682 server_base.cc:1034] running on GCE node
I20260807 04:28:08.369675 25682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:08.370940 25682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:08.372174 25682 hybrid_clock.cc:648] HybridClock initialized: now 1786076888372105 us; error 46 us; skew 500 ppm
I20260807 04:28:08.372496 25682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:08.375506 25682 init.cc:377] Logged in from keytab as kudu/127.1.244.253@KRBTEST.COM (short username kudu)
I20260807 04:28:08.376873 25682 webserver.cc:460] Webserver started at http://127.1.244.253:37409/ using document root <none> and password file <none>
I20260807 04:28:08.377542 25682 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:08.377651 25682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:08.377980 25682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:08.380940 25682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data/instance:
uuid: "ecbfd57e7b2d458194b6d40bf8c7570c"
format_stamp: "Formatted at 2026-08-07 04:28:08 on dist-test-slave-zkpd"
server_key: "2bb7550e749d23de0fa8cf35616fabab"
server_key_iv: "045d2c47fdf5e5c2ea8c5000a43f7826"
server_key_version: "encryptionkey@0"
I20260807 04:28:08.381716 25682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal/instance:
uuid: "ecbfd57e7b2d458194b6d40bf8c7570c"
format_stamp: "Formatted at 2026-08-07 04:28:08 on dist-test-slave-zkpd"
server_key: "2bb7550e749d23de0fa8cf35616fabab"
server_key_iv: "045d2c47fdf5e5c2ea8c5000a43f7826"
server_key_version: "encryptionkey@0"
I20260807 04:28:08.386822 25682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:28:08.390167 25697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:08.391431 25682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 04:28:08.391603 25682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal
uuid: "ecbfd57e7b2d458194b6d40bf8c7570c"
format_stamp: "Formatted at 2026-08-07 04:28:08 on dist-test-slave-zkpd"
server_key: "2bb7550e749d23de0fa8cf35616fabab"
server_key_iv: "045d2c47fdf5e5c2ea8c5000a43f7826"
server_key_version: "encryptionkey@0"
I20260807 04:28:08.391815 25682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:08.409214 25682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:08.412910 25682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:08.413167 25682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:08.432402 25682 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.253:35089
I20260807 04:28:08.432428 25750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.253:35089 every 8 connection(s)
I20260807 04:28:08.433792 25682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/data/info.pb
I20260807 04:28:08.437480 25751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:08.438395 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25682
I20260807 04:28:08.438596 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-1/wal/instance
I20260807 04:28:08.439035 2003 external_mini_cluster.cc:1428] Setting key 019d7f245eb709f42582e51f4b458181
I20260807 04:28:08.444103 25751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:28:08.471264 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.446718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.1.244.254:40015)
0807 04:28:08.446888 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.446893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.448271 (+ 1378us) server_negotiation.cc:407] Connection header received
0807 04:28:08.450129 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.450136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.450228 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.450337 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.451857 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.452610 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.454201 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.454602 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.454699 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.463790 (+ 9091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:08.463800 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:08.463808 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:08.463857 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:08.467766 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.468636 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.468644 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.468651 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.468826 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.469240 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.469248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.469253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.469765 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:08.469952 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:08.470627 (+ 675us) server_negotiation.cc:299] Negotiation successful
0807 04:28:08.470887 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 04:28:08.471971 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.447299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40015 (local address 127.0.0.1:34820)
0807 04:28:08.448110 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.448155 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:08.449852 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:08.450570 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:08.450577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:08.451084 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:08.451644 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.451674 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.452779 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.452786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.454054 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.454068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.454824 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.454833 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.455193 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.460755 (+ 5562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:08.460790 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:08.463532 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:08.467957 (+ 4425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.467967 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.467987 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.468492 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.468944 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.468947 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.468949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.469113 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.469908 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:08.469914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:08.470232 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 04:28:08.470452 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 04:28:08.470843 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":316,"threads_started":1}
WARNING: no policy specified for kudu/127.1.244.252@KRBTEST.COM; defaulting to no policy
I20260807 04:28:08.473800 25751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.244.252@KRBTEST.COM" created.
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:08.489188 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.474372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37976)
0807 04:28:08.474610 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.474628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:08.474726 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:08.475653 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:08.475659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:08.476175 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:08.476391 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.476406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.477860 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.477868 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.479356 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.479374 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.480279 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.480291 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.480493 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.481048 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:08.481073 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:08.483592 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:08.486636 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.486643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.486650 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.487209 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.487584 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.487591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.487598 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.487775 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.488497 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:08.488500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:08.488606 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 04:28:08.488732 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 04:28:08.488939 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 04:28:08.489697 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.474629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.1.244.253:35089)
0807 04:28:08.475199 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.475205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.475230 (+ 25us) server_negotiation.cc:407] Connection header received
0807 04:28:08.475270 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.475276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.475501 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.475613 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.476572 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.477727 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.479488 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.480117 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.480211 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.483798 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:08.483808 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:08.483819 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:08.483850 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:08.486504 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.487337 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.487343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.487345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.487462 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.487881 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.487907 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.487911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.488361 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:08.488594 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:08.488930 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 04:28:08.489222 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
I20260807 04:28:08.492161 25751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:08.493448 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.492881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43172)
0807 04:28:08.493089 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.493228 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 04:28:08.493781 25700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.495541 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.495036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43176)
0807 04:28:08.495251 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.495367 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 04:28:08.495855 25751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
Entry for principal kudu/127.1.244.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab.
I20260807 04:28:08.513137 2003 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
/tmp/dist-test-taskA8s8yC/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43845
--webserver_interface=127.1.244.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.244.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.244.254:40015,127.1.244.253:35089,127.1.244.252:43845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 04:28:08.545218 25751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 1
I20260807 04:28:08.546543 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.545960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43178)
0807 04:28:08.546196 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.546318 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:28:08.547662 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.547249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43190)
0807 04:28:08.547435 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.547530 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 04:28:08.547914 25751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.616132 25751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 2
I20260807 04:28:08.617594 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.616961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43192)
0807 04:28:08.617274 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.617383 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260807 04:28:08.618659 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.618279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43198)
0807 04:28:08.618414 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.618502 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 04:28:08.618901 25751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.635954 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } attempt: 5
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:08.650488 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.636927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37984 (local address 127.1.244.253:35089)
0807 04:28:08.637113 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.637118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.637159 (+ 41us) server_negotiation.cc:407] Connection header received
0807 04:28:08.637216 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.637220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.637293 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.637379 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.638363 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.639040 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.640775 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.641227 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.641319 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.645146 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:08.645151 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:08.645154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:08.645187 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:08.647882 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.648589 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.648595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.648599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.648724 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.649102 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.649108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.649112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.649402 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:08.649556 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:08.649941 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 04:28:08.650148 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 04:28:08.650489 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.636704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37984)
0807 04:28:08.636925 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.636947 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:08.637086 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:08.637453 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:08.637458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:08.637972 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:08.638211 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.638224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.639176 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.639182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.640594 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.640609 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.641428 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.641436 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.641655 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.642256 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:08.642281 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:08.644932 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:08.648062 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.648068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.648072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.648469 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.648842 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.648849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.648853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.648991 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.649522 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:08.649527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:08.649689 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 04:28:08.649843 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 04:28:08.650115 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 04:28:08.651571 25675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:08.652751 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.652351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43210)
0807 04:28:08.652522 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.652605 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 04:28:08.653717 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.653418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43222)
0807 04:28:08.653557 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.653619 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 04:28:08.653987 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.705216 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 1
I20260807 04:28:08.706422 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.705950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43234)
0807 04:28:08.706140 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.706231 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 04:28:08.707389 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.707112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43240)
0807 04:28:08.707231 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.707292 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:28:08.707638 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.710103 25751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 3
I20260807 04:28:08.710987 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.710579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43252)
0807 04:28:08.710754 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.710855 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 04:28:08.711987 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.711552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43260)
0807 04:28:08.711753 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.711830 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260807 04:28:08.712167 25751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.775980 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 2
I20260807 04:28:08.777369 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.776803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43266)
0807 04:28:08.777040 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.777130 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 04:28:08.778373 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.778058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43272)
0807 04:28:08.778196 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.778272 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 04:28:08.778599 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.855446 25751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 4
I20260807 04:28:08.856611 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.856134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43282)
0807 04:28:08.856371 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.856454 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 04:28:08.857532 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.857224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43288)
0807 04:28:08.857366 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.857437 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 04:28:08.857734 25751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
W20260807 04:28:08.867817 25759 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 04:28:08.868218 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 04:28:08.868360 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 04:28:08.868505 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 04:28:08.875563 25759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 04:28:08.875674 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 04:28:08.875818 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 04:28:08.875928 25759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 04:28:08.876022 25759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 04:28:08.884806 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 3
I20260807 04:28:08.885882 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.885426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43294)
0807 04:28:08.885646 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.885726 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 04:28:08.886116 25759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.244.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.244.254:40015,127.1.244.253:35089,127.1.244.252:43845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/krb5kdc/kudu_127.1.244.252.keytab
--principal=kudu/127.1.244.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.244.252:43845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.244.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 02:54:14 UTC on ccycloud.quasar-eedayb.root.comops.site
build id 4571743
ASAN enabled
I20260807 04:28:08.886889 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.886532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43310)
0807 04:28:08.886714 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.886791 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260807 04:28:08.887094 25675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.244.252:43845: Network error: Client connection negotiation failed: client connection to 127.1.244.252:43845: connect: Connection refused (error 111)
I20260807 04:28:08.887446 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 04:28:08.888741 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 04:28:08.900408 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:08.900492 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 04:28:08.900457 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 04:28:08.901615 25759 server_base.cc:1034] running on GCE node
I20260807 04:28:08.902374 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 04:28:08.903653 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 04:28:08.904866 25759 hybrid_clock.cc:648] HybridClock initialized: now 1786076888904769 us; error 63 us; skew 500 ppm
I20260807 04:28:08.905207 25759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 04:28:08.908377 25759 init.cc:377] Logged in from keytab as kudu/127.1.244.252@KRBTEST.COM (short username kudu)
I20260807 04:28:08.909780 25759 webserver.cc:460] Webserver started at http://127.1.244.252:41331/ using document root <none> and password file <none>
I20260807 04:28:08.910449 25759 fs_manager.cc:359] Metadata directory not provided
I20260807 04:28:08.910573 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 04:28:08.910924 25759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 04:28:08.913843 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data/instance:
uuid: "551fb701d8684021b1741345637ac176"
format_stamp: "Formatted at 2026-08-07 04:28:08 on dist-test-slave-zkpd"
server_key: "845d77bf4c390119385227a4a3b77d80"
server_key_iv: "cacf3353a65847a724ab74b8c49b81f6"
server_key_version: "encryptionkey@0"
I20260807 04:28:08.914630 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal/instance:
uuid: "551fb701d8684021b1741345637ac176"
format_stamp: "Formatted at 2026-08-07 04:28:08 on dist-test-slave-zkpd"
server_key: "845d77bf4c390119385227a4a3b77d80"
server_key_iv: "cacf3353a65847a724ab74b8c49b81f6"
server_key_version: "encryptionkey@0"
I20260807 04:28:08.919610 25759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 04:28:08.922820 25775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 04:28:08.924129 25759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 04:28:08.924319 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data,/tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal
uuid: "551fb701d8684021b1741345637ac176"
format_stamp: "Formatted at 2026-08-07 04:28:08 on dist-test-slave-zkpd"
server_key: "845d77bf4c390119385227a4a3b77d80"
server_key_iv: "cacf3353a65847a724ab74b8c49b81f6"
server_key_version: "encryptionkey@0"
I20260807 04:28:08.924535 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 04:28:08.939857 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 04:28:08.943867 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 04:28:08.944187 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 04:28:08.962210 25759 rpc_server.cc:307] RPC server started. Bound to: 127.1.244.252:43845
I20260807 04:28:08.962182 25829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.244.252:43845 every 8 connection(s)
I20260807 04:28:08.963469 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/data/info.pb
I20260807 04:28:08.967156 25830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 04:28:08.972491 2003 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu as pid 25759
I20260807 04:28:08.972687 2003 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskA8s8yC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786076526463677-2003-0/minicluster-data/master-2/wal/instance
I20260807 04:28:08.973121 2003 external_mini_cluster.cc:1428] Setting key ae775d9566132b3312780d8e899d57aa
I20260807 04:28:08.973591 25830 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
Aug 07 04:28:08 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260807 04:28:08.993507 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.976424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.1.244.254:40015)
0807 04:28:08.976955 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.976963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.977569 (+ 606us) server_negotiation.cc:407] Connection header received
0807 04:28:08.978554 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.978561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.978673 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.978791 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.980073 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.980792 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.982497 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.983011 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.983103 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.986926 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:08.986935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:08.986940 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:08.986988 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:08.990055 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.990888 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.990894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.990897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.991050 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.991491 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.991496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.991499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.991842 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:08.992033 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:08.992841 (+ 808us) server_negotiation.cc:299] Negotiation successful
0807 04:28:08.993070 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":268,"threads_started":1}
I20260807 04:28:08.994563 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.976527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.254:40015 (local address 127.0.0.1:34834)
0807 04:28:08.977393 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.977426 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:08.978265 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:08.978880 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:08.978886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:08.979512 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:08.979844 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.979861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.980984 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.980990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.982352 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.982364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.983227 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.983235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:08.983528 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.984446 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:08.984467 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:08.986724 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:08.990181 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.990194 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.990206 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.990768 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.991170 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:08.991175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:08.991178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:08.991373 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:08.992025 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:08.992032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:08.992416 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 04:28:08.992723 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 04:28:08.993201 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":65,"spinlock_wait_cycles":16000,"thread_start_us":372,"threads_started":1}
I20260807 04:28:08.994541 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.974902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.1.244.252:43845)
0807 04:28:08.975727 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.975737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.975772 (+ 35us) server_negotiation.cc:407] Connection header received
0807 04:28:08.978258 (+ 2486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.978278 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.978508 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.978662 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.980079 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.981413 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.982702 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.983298 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.983514 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:08.987644 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:08.987651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:08.987661 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:08.987687 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:08.990467 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.991175 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.991182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.991186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.991371 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:08.991748 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:08.991751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:08.991753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:08.992424 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:08.992749 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:08.993035 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 04:28:08.993296 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":604,"spinlock_wait_cycles":13568,"thread_start_us":370,"threads_started":1}
I20260807 04:28:08.996320 25830 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 04:28:09 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
Aug 07 04:28:09 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.254@KRBTEST.COM
I20260807 04:28:09.010488 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.997005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.253:35089 (local address 127.0.0.1:37990)
0807 04:28:08.997149 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:08.997166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:08.997273 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:08.997738 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:08.997743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:08.998214 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:08.998380 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:08.998389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.999761 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:08.999767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.000967 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.000978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.001964 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.001968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.002106 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.002611 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:09.002635 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:09.004782 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:09.008149 (+ 3367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.008152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.008155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.008496 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.008850 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.008856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.008859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.009009 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.009555 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:09.009560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:09.009700 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 04:28:09.009864 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 04:28:09.010139 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 04:28:09.010545 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.997216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.1.244.253:35089)
0807 04:28:08.997419 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.997425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.997444 (+ 19us) server_negotiation.cc:407] Connection header received
0807 04:28:08.997505 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.997511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.997609 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.997713 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.998531 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:08.999638 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.001173 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.001831 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.001970 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.004996 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.005005 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.005011 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.005060 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.007989 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.008598 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.008602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.008604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.008727 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.009121 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.009139 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.009143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.009415 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.009589 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.009961 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.010158 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 04:28:09.011399 25830 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } has no permanent_uuid. Determining permanent_uuid...
I20260807 04:28:09.013550 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:08.997587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.1.244.254:40015)
0807 04:28:08.997738 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:08.997743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:08.997796 (+ 53us) server_negotiation.cc:407] Connection header received
0807 04:28:08.997973 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:08.997977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:08.998066 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:08.998165 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:08.999565 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.000582 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.002246 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.002900 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.003046 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.007374 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.007379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.007382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.007411 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.010538 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.011376 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.011383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.011386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.011537 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.011908 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.011915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.011922 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.012285 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.012557 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.013129 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.013291 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 04:28:09 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.252@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
Aug 07 04:28:09 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.244.253@KRBTEST.COM
I20260807 04:28:09.024914 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.012229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43322)
0807 04:28:09.012391 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:09.012408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:09.012508 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:09.012778 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:09.012783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:09.013243 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:09.013430 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.013445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.014747 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.014750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.015749 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.015761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.016728 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.016732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.017045 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.017575 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:09.017590 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:09.019536 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:09.022548 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.022552 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.022556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.022914 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.023297 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.023303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.023306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.023434 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.023942 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:09.023946 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:09.024088 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 04:28:09.024223 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 04:28:09.024515 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":19}
I20260807 04:28:09.024986 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.012152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.1.244.252:43845)
0807 04:28:09.012327 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.012333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.012452 (+ 119us) server_negotiation.cc:407] Connection header received
0807 04:28:09.012597 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.012600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.012648 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.012718 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.013568 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.014623 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.015885 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.016589 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.016729 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.019706 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.019712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.019716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.019753 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.022417 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.023027 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.023031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.023035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.023197 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.023546 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.023553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.023556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.023812 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.023992 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.024308 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.024660 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":170,"server-negotiator.queue_time_us":60}
I20260807 04:28:09.029762 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.015177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.1.244.253:35089)
0807 04:28:09.015353 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.015358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.015375 (+ 17us) server_negotiation.cc:407] Connection header received
0807 04:28:09.015476 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.015481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.015543 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.015644 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.017138 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.018015 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.019562 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.020045 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.020178 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.023956 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.023962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.023966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.024004 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.027027 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.027714 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.027722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.027726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.027920 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.028377 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.028383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.028387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.028739 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.028996 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.029202 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.029413 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 04:28:09.031065 25830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: Bootstrap starting.
I20260807 04:28:09.034345 2003 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 04:28:09.034718 25830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: Neither blocks nor log segments found. Creating new log.
I20260807 04:28:09.036012 25830 log.cc:824] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:09.038805 25830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: No bootstrap required, opened a new log
I20260807 04:28:09.037232 25650 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "80a32deba3724097baa35484dac26305"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34840
I20260807 04:28:09.045780 25830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.046118 25830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551fb701d8684021b1741345637ac176, State: Initialized, Role: FOLLOWER
I20260807 04:28:09.046540 25830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.047505 25837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.047818 25837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.048772 25830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 04:28:09.058411 25849 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 04:28:09.058557 25849 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 04:28:09.055884 25759 master_runner.cc:186] Error getting master registration for 127.1.244.254:40015: OK, instance_id { permanent_uuid: "80a32deba3724097baa35484dac26305" instance_seqno: 1786076887921217 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:28:09.064064 25759 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:28:09.064314 25675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 4
Aug 07 04:28:09 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076887, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.254@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260807 04:28:09.081303 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.064989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43326)
0807 04:28:09.065242 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:09.065261 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:09.065396 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:09.065748 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:09.065754 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:09.066253 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:09.066471 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.066484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.067949 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.067955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.069505 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.069521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.070629 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.070635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.070835 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.071527 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:09.071552 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:09.074467 (+ 2915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:09.078499 (+ 4032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.078507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.078514 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.079040 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.079431 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.079435 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.079438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.079589 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.080320 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:09.080326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:09.080511 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 04:28:09.080657 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 04:28:09.080914 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260807 04:28:09.081346 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.065028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.1.244.252:43845)
0807 04:28:09.065199 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.065205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.065319 (+ 114us) server_negotiation.cc:407] Connection header received
0807 04:28:09.065518 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.065523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.065596 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.065702 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.066614 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.067766 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.069671 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.070475 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.070592 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.074682 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.074689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.074694 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.074729 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.078357 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.079157 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.079163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.079167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.079302 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.079702 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.079707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.079711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.080178 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.080315 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.080767 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.081002 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 04:28:09.087316 25675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305: Bootstrap starting.
I20260807 04:28:09.091239 25675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305: Neither blocks nor log segments found. Creating new log.
I20260807 04:28:09.092427 25675 log.cc:824] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:09.095522 25675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305: No bootstrap required, opened a new log
I20260807 04:28:09.100939 25675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.101388 25675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a32deba3724097baa35484dac26305, State: Initialized, Role: FOLLOWER
I20260807 04:28:09.101897 25675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.103001 25853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.103360 25853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.104163 25675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:28:09.104367 25650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 04:28:09.104607 25650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:09.107981 25650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.110486 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80a32deba3724097baa35484dac26305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "551fb701d8684021b1741345637ac176"
I20260807 04:28:09.111047 25803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:09.111266 25650 leader_election.cc:288] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [CANDIDATE]: Term 1 election: Requested vote from peers ecbfd57e7b2d458194b6d40bf8c7570c (127.1.244.253:35089), 551fb701d8684021b1741345637ac176 (127.1.244.252:43845)
I20260807 04:28:09.114708 25803 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a32deba3724097baa35484dac26305 in term 1.
I20260807 04:28:09.115367 25624 leader_election.cc:302] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551fb701d8684021b1741345637ac176, 80a32deba3724097baa35484dac26305; no voters:
I20260807 04:28:09.115819 25853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 04:28:09.126351 25853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [term 1 LEADER]: Becoming Leader. State: Replica: 80a32deba3724097baa35484dac26305, State: Running, Role: LEADER
I20260807 04:28:09.126953 25853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.134794 25858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a32deba3724097baa35484dac26305. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.135183 25858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:09.139369 25862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 04:28:09.150509 25862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 04:28:09.156912 25751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } attempt: 5
W20260807 04:28:09.160905 25627 tablet.cc:2369] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 04:28:09 dist-test-slave-zkpd krb5kdc[25593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786076888, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.244.253@KRBTEST.COM for kudu/127.1.244.252@KRBTEST.COM
I20260807 04:28:09.169732 25803 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [term 1 FOLLOWER]: Refusing update from remote peer 80a32deba3724097baa35484dac26305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:28:09.170897 25858 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:28:09.175482 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.157535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.244.252:43845 (local address 127.0.0.1:43334)
0807 04:28:09.157942 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 04:28:09.157965 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 04:28:09.158087 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 04:28:09.159008 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 04:28:09.159014 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 04:28:09.159558 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 04:28:09.159772 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.159787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.161280 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.161285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.163014 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 04:28:09.163031 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.164507 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.164513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 04:28:09.164745 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.165517 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 04:28:09.165539 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 04:28:09.168420 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 04:28:09.172573 (+ 4153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.172578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.172582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.173114 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.173578 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 04:28:09.173583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 04:28:09.173587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 04:28:09.173772 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 04:28:09.174473 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 04:28:09.174477 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 04:28:09.174636 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 04:28:09.174781 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 04:28:09.175096 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260807 04:28:09.176682 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.158349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.1.244.252:43845)
0807 04:28:09.158525 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.158531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.158549 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:28:09.158611 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.158616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.158702 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.158797 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.159985 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.161117 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.163205 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.163819 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.164302 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.168630 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.168635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.168639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.168677 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.172417 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.173255 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.173261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.173267 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.173441 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.173922 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.173926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.173929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.174245 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.174395 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.175840 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.176187 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 04:28:09.182643 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: Bootstrap starting.
I20260807 04:28:09.187274 25751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: Neither blocks nor log segments found. Creating new log.
W20260807 04:28:09.196035 25781 tablet.cc:2369] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 04:28:09.195902 25837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a32deba3724097baa35484dac26305. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.196255 25837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.197355 25751 log.cc:824] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: Log is configured to *not* fsync() on all Append() calls
I20260807 04:28:09.204417 25869 mvcc.cc:205] Tried to move back new op lower bound from 7315770938020421632 to 7315770937870127104. Current Snapshot: MvccSnapshot[applied={T|T < 7315770938020421632}]
I20260807 04:28:09.205992 25751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: No bootstrap required, opened a new log
I20260807 04:28:09.220638 25837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.220916 25837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.227918 25853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.228252 25853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:09.229027 25862 catalog_manager.cc:1347] Generated new cluster ID: 4b15ce9d034e43959612582f562783eb
I20260807 04:28:09.229115 25862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 04:28:09.234655 25751 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.235009 25751 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbfd57e7b2d458194b6d40bf8c7570c, State: Initialized, Role: FOLLOWER
I20260807 04:28:09.235466 25751 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } }
I20260807 04:28:09.251080 25606 master_runner.cc:186] Error getting master registration for 127.1.244.253:35089: OK, instance_id { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" instance_seqno: 1786076888426186 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 04:28:09.266996 25606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 04:28:09.272317 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.272861 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.273811 25751 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 04:28:09.274940 25724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [term 0 FOLLOWER]: Advancing to term 1
I20260807 04:28:09.277457 25725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80a32deba3724097baa35484dac26305" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c"
I20260807 04:28:09.278038 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.201809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.1.244.252:43845)
0807 04:28:09.202145 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.202150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.225971 (+ 23821us) server_negotiation.cc:407] Connection header received
0807 04:28:09.226361 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.226367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.226446 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.226515 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.244283 (+ 17768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.245336 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.260668 (+ 15332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.261384 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.261548 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.270842 (+ 9294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.270849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.270854 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.270897 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.274411 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.275219 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.275223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.275227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.275389 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.275797 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.275802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.275805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.276177 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.276417 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.277371 (+ 954us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.277651 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260807 04:28:09.278436 25724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [term 1 FOLLOWER]: Refusing update from remote peer 80a32deba3724097baa35484dac26305: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 04:28:09.281029 25858 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 04:28:09.282210 25725 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a32deba3724097baa35484dac26305 in term 1.
I20260807 04:28:09.285107 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.201641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.1.244.254:40015)
0807 04:28:09.202014 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.202021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.221202 (+ 19181us) server_negotiation.cc:407] Connection header received
0807 04:28:09.222611 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.222617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.222700 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.222772 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.240999 (+ 18227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.242215 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.260742 (+ 18527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.261437 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.261565 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.267290 (+ 5725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.267297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.267303 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.267344 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.272324 (+ 4980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.273136 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.273141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.273144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.273318 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.275479 (+ 2161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.275486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.275489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.275934 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.276180 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.276740 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.276999 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260807 04:28:09.293826 25862 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 04:28:09.294011 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:09.221781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.1.244.253:35089)
0807 04:28:09.221956 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.221962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.247946 (+ 25984us) server_negotiation.cc:407] Connection header received
0807 04:28:09.261780 (+ 13834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.261786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.261878 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.261952 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.269379 (+ 7427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.270574 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.277986 (+ 7412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:09.278714 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:09.278853 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:09.286654 (+ 7801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 04:28:09.286661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 04:28:09.286667 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 04:28:09.286712 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 04:28:09.290513 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.291371 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.291376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.291379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.291553 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 04:28:09.292083 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 04:28:09.292088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 04:28:09.292092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 04:28:09.292489 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 04:28:09.292721 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:09.293358 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 04:28:09.293603 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 04:28:09.295604 25862 catalog_manager.cc:1504] Loading token signing keys...
I20260807 04:28:09.296730 25898 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 04:28:09.296856 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80a32deba3724097baa35484dac26305. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.297134 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.297489 25896 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.297789 25896 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c [sys.catalog]: This master's current role is: FOLLOWER
I20260807 04:28:09.302443 25853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80a32deba3724097baa35484dac26305" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a32deba3724097baa35484dac26305" member_type: VOTER last_known_addr { host: "127.1.244.254" port: 40015 } } peers { permanent_uuid: "ecbfd57e7b2d458194b6d40bf8c7570c" member_type: VOTER last_known_addr { host: "127.1.244.253" port: 35089 } } peers { permanent_uuid: "551fb701d8684021b1741345637ac176" member_type: VOTER last_known_addr { host: "127.1.244.252" port: 43845 } } }
I20260807 04:28:09.302706 25853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305 [sys.catalog]: This master's current role is: LEADER
I20260807 04:28:09.314750 25862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80a32deba3724097baa35484dac26305: Generated new TSK 0
I20260807 04:28:09.315640 25862 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 04:28:09.415283 25833 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.399595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.1.244.252:43845)
0807 04:28:09.400000 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.400006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.412526 (+ 12520us) server_negotiation.cc:407] Connection header received
0807 04:28:09.412746 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.412752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.412838 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.412924 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.415017 (+ 2093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43360: BlockingRecv error: recv got EOF from 127.0.0.1:43360 (error 108)
Metrics: {"server-negotiator.queue_time_us":270}
W20260807 04:28:09.419332 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.413253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.1.244.253:35089)
0807 04:28:09.413876 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.413882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.417736 (+ 3854us) server_negotiation.cc:407] Connection header received
0807 04:28:09.417988 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.417993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.418072 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.418133 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.419106 (+ 973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38014: BlockingRecv error: recv got EOF from 127.0.0.1:38014 (error 108)
Metrics: {"server-negotiator.queue_time_us":490}
W20260807 04:28:09.421060 25834 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.399401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.1.244.254:40015)
0807 04:28:09.399842 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.399849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.412012 (+ 12163us) server_negotiation.cc:407] Connection header received
0807 04:28:09.413400 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.413405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.413500 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.413597 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.420837 (+ 7240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34856: BlockingRecv error: recv got EOF from 127.0.0.1:34856 (error 108)
Metrics: {"server-negotiator.queue_time_us":281}
W20260807 04:28:09.427666 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.425630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.1.244.253:35089)
0807 04:28:09.425778 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.425783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.426240 (+ 457us) server_negotiation.cc:407] Connection header received
0807 04:28:09.426471 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.426477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.426588 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.426695 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.427404 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38028: BlockingRecv error: recv got EOF from 127.0.0.1:38028 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 04:28:09.477977 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.468287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.1.244.253:35089)
0807 04:28:09.468488 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.468494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.476496 (+ 8002us) server_negotiation.cc:407] Connection header received
0807 04:28:09.476733 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.476738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.476825 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.476941 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.477811 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38040: BlockingRecv error: recv got EOF from 127.0.0.1:38040 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 04:28:09.489809 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.483969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.1.244.253:35089)
0807 04:28:09.484154 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.484158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.488494 (+ 4336us) server_negotiation.cc:407] Connection header received
0807 04:28:09.488688 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.488694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.488776 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.488881 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.489598 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38054: BlockingRecv error: recv got EOF from 127.0.0.1:38054 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 04:28:09.503342 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.501204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.1.244.253:35089)
0807 04:28:09.501452 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.501460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.502036 (+ 576us) server_negotiation.cc:407] Connection header received
0807 04:28:09.502221 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.502233 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.502354 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.502472 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.503150 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38064: BlockingRecv error: recv got EOF from 127.0.0.1:38064 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
W20260807 04:28:09.516578 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.514603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.1.244.253:35089)
0807 04:28:09.514808 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.514816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.515268 (+ 452us) server_negotiation.cc:407] Connection header received
0807 04:28:09.515464 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.515469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.515542 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.515642 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.516349 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38080: BlockingRecv error: recv got EOF from 127.0.0.1:38080 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 04:28:09.540655 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.538370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.1.244.253:35089)
0807 04:28:09.538580 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.538588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.539110 (+ 522us) server_negotiation.cc:407] Connection header received
0807 04:28:09.539308 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.539314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.539415 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.539537 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.540428 (+ 891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38094: BlockingRecv error: recv got EOF from 127.0.0.1:38094 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 04:28:09.580017 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.577468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38098 (local address 127.1.244.253:35089)
0807 04:28:09.577685 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.577691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.578282 (+ 591us) server_negotiation.cc:407] Connection header received
0807 04:28:09.578516 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.578522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.578630 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.578795 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.579692 (+ 897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38098: BlockingRecv error: recv got EOF from 127.0.0.1:38098 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 04:28:09.653751 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.651284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.1.244.253:35089)
0807 04:28:09.651538 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.651545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.652187 (+ 642us) server_negotiation.cc:407] Connection header received
0807 04:28:09.652401 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.652407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.652499 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.652607 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.653497 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38114: BlockingRecv error: recv got EOF from 127.0.0.1:38114 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260807 04:28:09.787287 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:09.785232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.1.244.253:35089)
0807 04:28:09.785483 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:09.785489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:09.785797 (+ 308us) server_negotiation.cc:407] Connection header received
0807 04:28:09.786029 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:09.786039 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:09.786111 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:09.786224 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:09.787027 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38118: BlockingRecv error: recv got EOF from 127.0.0.1:38118 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 04:28:10.051170 25755 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 04:28:10.049404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.1.244.253:35089)
0807 04:28:10.049632 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:10.049639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:10.049948 (+ 309us) server_negotiation.cc:407] Connection header received
0807 04:28:10.050100 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:10.050105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:10.050178 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:10.050310 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 04:28:10.050953 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38120: BlockingRecv error: recv got EOF from 127.0.0.1:38120 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 04:28:10.060477 25849 catalog_manager.cc:1259] Loaded cluster ID: 4b15ce9d034e43959612582f562783eb
I20260807 04:28:10.060561 25849 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: loading cluster ID for follower catalog manager: success
I20260807 04:28:10.063040 25849 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: acquiring CA information for follower catalog manager: success
I20260807 04:28:10.064843 25849 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 551fb701d8684021b1741345637ac176: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:28:10.301255 25898 catalog_manager.cc:1259] Loaded cluster ID: 4b15ce9d034e43959612582f562783eb
I20260807 04:28:10.301442 25898 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: loading cluster ID for follower catalog manager: success
I20260807 04:28:10.304127 25898 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: acquiring CA information for follower catalog manager: success
I20260807 04:28:10.306143 25898 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ecbfd57e7b2d458194b6d40bf8c7570c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 04:28:10.583650 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:10.575607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.1.244.253:35089)
0807 04:28:10.576388 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:10.576395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:10.576697 (+ 302us) server_negotiation.cc:407] Connection header received
0807 04:28:10.576935 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:10.576939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:10.577176 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:10.577288 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:10.577910 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.578962 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.581012 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.581545 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.581656 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:10.582287 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:10.582934 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:10.583039 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:10.583205 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 04:28:10.583269 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":431,"threads_started":1}
I20260807 04:28:10.597153 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:10.590289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.1.244.252:43845)
0807 04:28:10.590927 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:10.590932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:10.590950 (+ 18us) server_negotiation.cc:407] Connection header received
0807 04:28:10.591039 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:10.591043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:10.591182 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:10.591317 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:10.591777 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.592586 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.594574 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.595017 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.595162 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:10.595811 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:10.596473 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:10.596578 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:10.596742 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 04:28:10.596808 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":407,"threads_started":1}
I20260807 04:28:10.622385 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:10.603916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.1.244.254:40015)
0807 04:28:10.613582 (+ 9666us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:10.613588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:10.614465 (+ 877us) server_negotiation.cc:407] Connection header received
0807 04:28:10.614599 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:10.614605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:10.614822 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:10.615046 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:10.615991 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.617150 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.619380 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.620024 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.620141 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:10.620826 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:10.621258 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:10.621355 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:10.621999 (+ 644us) server_negotiation.cc:299] Negotiation successful
0807 04:28:10.622088 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9522,"thread_start_us":9670,"threads_started":1}
I20260807 04:28:10.625427 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:10.604436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.1.244.252:43845)
0807 04:28:10.609821 (+ 5385us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:10.609838 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:10.614662 (+ 4824us) server_negotiation.cc:407] Connection header received
0807 04:28:10.614948 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:10.614954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:10.615068 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:10.615131 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:10.616225 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.617650 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.622156 (+ 4506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.623043 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.623163 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:10.624041 (+ 878us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:10.624206 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:10.624290 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:10.624912 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 04:28:10.625021 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5219}
I20260807 04:28:10.629328 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 04:28:10.614765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.1.244.253:35089)
0807 04:28:10.614957 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 04:28:10.614963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 04:28:10.621900 (+ 6937us) server_negotiation.cc:407] Connection header received
0807 04:28:10.623093 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 04:28:10.623100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 04:28:10.623180 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 04:28:10.623235 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 04:28:10.623790 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.624857 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.627074 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 04:28:10.627729 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 04:28:10.627862 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 04:28:10.628438 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 04:28:10.628587 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 04:28:10.628672 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 04:28:10.628873 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 04:28:10.628965 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 04:28:10.633481 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25606
I20260807 04:28:10.634474 25670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:28:10.744225 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25606
I20260807 04:28:10.768898 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25682
I20260807 04:28:10.770030 25746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:28:10.875228 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25682
I20260807 04:28:10.897339 2003 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu pid 25759
I20260807 04:28:10.898527 25824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 04:28:11.004987 2003 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskA8s8yC/build/asan/bin/kudu with pid 25759
2026-08-07T04:28:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4950 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9890 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (364511 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST